Microsoft-prevalence-based analysis of the File Detection Test

Transcription

1 Commissioned by Microsoft Microsoft-prevalence-based analysis of the File Detection Test Supplementary Report to the File Detection Test of March 2014 Language: English March 2014 Last Revision: 7 th August 2014 Commissioned by Microsoft Commissioned by Microsoft - 1 -

2 Introduction This report has been commissioned by Microsoft and is based on telemetry data provided by Microsoft. This is a first prototype customer-impact report; more enhanced versions might be provided for future File-Detection Test reports. The essence of this report is as follows. Of the prevalent samples used in the malware-detection test, some will pose a greater threat to the average user than others as they are more widespread than others. Some may target e.g. a specific company or user base, but present less of risk to other users. Other malware samples may only be found on specific websites, affect specific countries/regions, or only be relevant to particular operating system versions or interface languages. Microsoft s initiative uses its global telemetry data (malware prevalence) to consider the customer impact posed by missed detections. That is, the malware files that antimalware products failed to detect are weighted based on malware-family prevalence, and each vendor s prevalence-weighted results are reported along with the file-detection results in this report. These results are designed to give greater insight into the customer impact of the missed detections during testing. In addition to global prevalence weighting impact, geo-location prevalence is also used to determine the customer impact of missed detections in specific countries for products tested. This is used to present the filedetection efficacy of antimalware products in the test against prevalent malware samples. This report is supplementary to AV-Comparatives main report 1, already published, of the March 2014 File-Detection Test. No additional testing has been performed; rather, the existing test results have been re-analysed from a different perspective, to consider what impact the missed samples are likely to have on customers. It is conceivable that a product with a lower score in the test may actually protect the average user better than one with a higher score, under specific circumstances. Let us imagine that Product A detects 99% of malware samples in the test, but that the 1% of samples not detected are very widespread, and that the average user is quite likely to encounter them. Product B, on the other hand, only detects 98% of samples, but the samples missed are either not as prevalent, or only run on a specific operating system. In this case, users would probably be more at risk using Product A, as it misses more of the malware that is likely to present a threat to them. AV-Comparatives has for many years focused on using prevalent samples in its tests, as mentioned in our reports and also in a Microsoft blog 2. Furthermore, same sample variants (e.g. polymorphic malware) are clustered to avoid a disproportional test-set 3. AV-Comparatives makes uses of telemetry data from various sources, not just Microsoft, as the test-set must remain independent and not based solely on data provided by one specific vendor or organisation. Therefore, minor discrepancies between one vendor s data and our independently sorted combination are possible. The original File- Detection Test in March 2014 used a malware set sorted using various telemetry sources; however, the analysis in this supplementary report is based solely on Microsoft s data Commissioned by Microsoft - 2 -

3 Tested products: The following products tested in March 2014 are included in this report: 1. AhnLab V3 Internet Security avast! Free Antivirus AVG Internet Security AVIRA Internet Security Baidu Antivirus 2014 (en) 6. Bitdefender Internet Security BullGuard Internet Security Emsisoft Anti-Malware escan Internet Security ESET Smart Security F-Secure Internet Security Fortinet FortiClient Kaspersky Internet Security Kingsoft Internet Security 2013.SP6 15. Lavasoft Ad-Aware Free Antivirus McAfee Internet Security Microsoft Security Essentials Panda Cloud Free Antivirus Qihoo 360 Internet Security 4.9 (en) 20. Sophos Endpoint Security Tencent QQ PC Manager ThreatTrack Vipre Internet Security Trend Micro Titanium Internet Security 2014 The test-set used was built consulting telemetry data from various sources (not only Microsoft), with the aim of including mainly prevalent malicious samples from the last weeks/months prior the test which posed a threat to users in the field. Detection vs. Protection Although very important, the file-detection rate of a product is only one aspect of a complete antivirus product. Almost all antivirus products contain features such as URL-blockers and behavioural protection that protect the user s computer without necessarily identifying every malicious file. AV-Comparatives also provides a whole-product dynamic real-world protection test 4, as well as other test reports that cover these aspects/features of the products. We invite users to look at our other tests and not only the File-Detection Test, even though a good file-detection rate is still one of the most important, deterministic and reliable basic features of an anti-virus product. 4 Commissioned by Microsoft - 3 -

4 Methodology This analysis was carried out using AV-Comparatives file-detection test data from March Telemetry data was gathered for the files in the test over the period between December and April This telemetry came from Microsoft real-time protection (RTP) products and included not only threat telemetry but also behaviour-based early warning telemetry. Only computers whose users have agreed to provide data to Microsoft are considered when calculating encounter rates. This data was used to generate a weighting to represent customer impact for the files used in the test. This weighting was generated by first counting the total number of distinct computers identified through a unique product GUID (not IP address) that encountered the malware files in the test. Furthermore, a weighting for the malware family associated with the specific files in the test was generated from computers running Microsoft RTP products to assess total customer impact for that family in March Family Weight Calculation The family weighting was calculated by dividing the number of Microsoft RTP computers that encountered each family by the entire number of RTP computers that encounter any high or severe malware during the period. The family mapping used is based on the naming 5 given by Microsoft at a specific point in time (time of testing). This weight was then applied to misses for that family in the test set (see calculation details below). Finally, these weightings were summed to give the final weighted score of each vendor. Descriptions and Information about malware families can be found in the Microsoft Malware Protection Center s Malware Encyclopedia Vendor impact calculation The total vendor impact calculation is generated by first measuring the machines encountering the missed files per vendor per family, which we call the MissedFileFamilyMachineCount. The FamilyWeight for each family is then applied to the MissedFileFamilyMachineCount and the results are summed over all families, which produces the NonNormalizedVendorImpact. 5 Commissioned by Microsoft - 4 -

5 The WholeTestImpact is then calculated to normalize the NonNormalizedVendorImpact. The WholeTestImpact is calculated by first measuring the machines encountering the all files per family in the test set, which we call the WholeTestFileFamilyMachineCount. The FamilyWeight for each family is then applied to the WholeTestFileFamilyMachineCount and the results are summed over all families, which produces the WholeTestImpact. Finally, the VendorImpact is calculated by dividing the NonNormalizedVendorImpact by the WholeTestImpact. Country Vendor Impact calculation Country-based vendor impacts were calculated in a similar way to the total vendor impact calculation outlined above: The MissedFileFamilyMachineCount per vendor per family from above was weighted using a the FamilyWeightByCountry, which is a geo-specific family weighting for each country and malware family. This weighted result produces the NonNormalizedVendorImpact ByCountry. Finally, the WholeTestImpactPerCountry was calculated by applying the FamilyWeightByCountry to the WholeTestFileMachineCount PerFamily described above. This was then used to normalize the NonNormalizedVendorImpact ByCountry to produce the final VendorImpactByCountry. Commissioned by Microsoft - 5 -

6 Microsoft generated telemetry information on many of the files that they did not have detection for (10% of the test-set was not detected by Microsoft), so for those files they had customer impact numbers as well. For the files for which Microsoft did not have customer impact numbers, Microsoft used the average customer impact of the low-prevalence files in the test. Therefore, a generic family has been created and an average family weighting has been used to calculate the impact of malware samples not detected by Microsoft. This report should be regarded as a prototype, the purpose of which is spark debate on the significance of prevalence data, and promote ideas for improving the method. For upcoming prevalence analysis reports, the goal would be to translate third-party detections into Microsoft families to increase the accuracy. This would involve other vendors working closely with Microsoft to understand how various vendors count prevalence and to investigate how to roll in telemetry from other vendors, in order to produce a better picture. AV-Comparatives encourage vendors to share their telemetry data with Microsoft, in order to get a more significant and impartial customer-impact analysis. Microsoft regularly reviews and refines its data collection methodology to improve its scope and accuracy. For this reason, the statistics presented in this report may differ from comparable statistics in future reports. Microsoft uses relativly few family names, of which about 1300 have been seen in-the-field, and about 600 with currently at least 1 machine reporting the family in question. The test-set of AV- Comparatives consisted of about 1250 different families according to Microsoft s family mapping. When Microsoft gains new information on malware samples and families, Microsoft s family-based detection may change. About 2% of Microsoft detected samples were later mapped to different families from those to which they had originally been assigned. Commissioned by Microsoft - 6 -

7 Test-Set description The test-set used in March 2014 for the File-Detection Test contained malware samples. The number of encounters caused by the malware samples used in the test was according to Microsoft s telemetry data around 3,361,802. Over 3 million unique machines were affected. The world map below shows the countries in which the malicious files had the biggest impact. There are over 150 countries of the world for which Microsoft have data for less than 100,000 computers. These are considered to be too small to be statistically relevant the margin of error is too high to accurately represent the population of Internet users in the country. These appear as white on the map. The impact on the remaining 91 countries (~94% of machine population) are shown as blue in the map. Top 30 impacted countries: 1. India 10.1% 2. Mexico 6.6% 3. Turkey 6.6% 4. United States 5.4% 5. Philippines 4.9% 6. Russian Federation 4.8% 7. Indonesia 4.8% 8. Brazil 3.7% 9. Pakistan 3.1% 10. Vietnam 2.6% 11. Ukraine 2.5% 12. Iran 1.8% 13. Colombia 1.8% 14. Argentinia 1.8% 15. Thailand 1.8% 16. Malaysia 1.6% 17. Peru 1.5% 18. United Kingdom 1.4% 19. Italy 1.3% 20. Egypt 1.3% 21. France 1.2% 22. Kazakhstan 1.1% 23. Venezuela 1.0% 24. Germany 0.9% 25. Algeria 0.8% 26. Ecuador 0.8% 27. Poland 0.8% 28. Canada 0.8% 29. Chile 0.8% 30. Saudi Arabia 0.8% Commissioned by Microsoft - 7 -

8 Distribution of Malware Types in the test-set Top 20 Malware Families in the test-set 1. Vobfus 6.5% 2. Zbot 5.5% 3. Bladabindi 4.2% 4. VBInject 4.1% 5. Urausy 4.0% 6. Injector 3.3% 7. CeeInject 3.3% 8. Dorkbot 2.9% 9. Sality 2.8% 10. Winwebsec 2.6% 11. Dynamer 2.5% 12. Expiro 2.3% 13. Siresef 2.1% 14. Ramnit 2.1% 15. Beebone 2.1% 16. Virut 1.6% 17. Malagent 1.5% 18. Gamarue 1.5% 19. Autorun 1.3% 20. Obfuscator 1.2% Top 20 Malware Families in the test-set with highest Customer Impact according to Microsoft Malware Family 1. Iethic 2. Dorkbot* 3. Gamarue* 4. Zbot* 5. Sirefef 6. Necurs* 7. Kilim* 8. Cutwail 9. [Undefined]* 10. Kuluoz Machine Count % of all machines 30% 20% 16% 7% 6% 5% 5% 4% 4% 4% Malware Family 11. Upatre 12. CeeInject 13. Fareit 14. Malagent* 15. Meredrop 16. VBInject 17. Bladabindi* 18. Caphaw 19. Dynamer* 20. Miuref Machine Count % of all machines 3% 3% 3% 3% 3% 3% 3% 3% 2% 2% The Top Missed Families in the Top20 Malware Families in the test-set with highest Customer Impact are marked with an asterisk. Commissioned by Microsoft - 8 -

9 Detection Rates and Customer Impact Based on the missed samples and the detection rate over the whole test-set, Microsoft have calculated the normalized Customer Impact. This can be seen in the table below. The different colors in the table illustrate products scoring better than the baseline (Microsoft). Customer Impact (normalized) Missed machines out of 1000 Missed Samples 6 100% -Missed Samples 1. Kaspersky Lab % 99.8% 2. McAfee % 99.3% 3. AVIRA % 99.2% 4. Tencent % 99.2% 5. Panda % 99.3% 6. ThreatTrack Vipre % 98.9% 7. Fortinet % 99.6% 8. Sophos % 98.3% 9. ESET % 98.8% 10. AVG % 97.5% 11. F-Secure % 99.6% 12. Trend Micro % 99.0% 13. escan % 99.6% 14. Bitdefender % 99.5% 15. Lavasoft % 99.5% 16. Kingsoft % 99.5% 17. Emsisoft % 99.5% 18. Qihoo % 99.5% 19. BullGuard % 99.5% 20. Avast % 97.7% 21. Baidu % 99.6% 22. AhnLab % 89.0% * Microsoft is represented as the baseline with the following results: customer impact (normalized) of , missed machines out of 1000, missed 10% of samples, or 100% - missed sample percentage of 90%. For some few vendors there is a significant difference between the number of missed samples and the Customer Impact. In some cases, a high number of missed files did not translate into a high Customer Impact. This indicates that the samples detected in the test were those with higher prevalence, while those that were not detected had lower prevalence. Conversely, in some cases vendors with a low number of missed samples had a relatively high Customer Impact. In these cases, the samples that were missed were higher prevalence samples, which had a greater impact on the customer population. This supplementary report suggests that using prevalence to weight malicious file detection tests can produce useful insights into the efficacy of antimalware products against the most common risks. That is, when file-detection test results are weighted by malware family prevalence it is possible to get a better idea of the risk that customers face when using antimalware products. Furthermore, it shows that detection of highly prevalent malware is valuable in reducing the risk to customers, and is a useful factor for detection prioritization. 6 About 77% of the test-set was detected by all products in the test, as the samples are well-known/common. 7 According to the data and analysis from Microsoft, Baidu has a high impact score, because in the few misses it had, it missed high-prevalence files that belonged to prevalent families this is why it looks like an outlier. Commissioned by Microsoft - 9 -

10 Customer Impact Report March 2014 Heat-Maps Overview The interactive heat maps can be found on TEST-SET AhnLab Avast AVG AVIRA Baidu Bitdefender BullGuard Emsisoft escan ESET Fortinet F-Secure Kaspersky Lab Kingsoft Lavasoft McAfee Microsoft Panda Qihoo Sophos Tencent ThreatTrack Vipre Trend Micro The heat maps for each vendor, i.e. the coloured maps of the world show data that is normalised by the relative size of the country. Thus the maps represent the countries with the highest risk relative to the prevalence of files that were missed in the test set. This normalisation differs from the heat map displayed in the Test-Set Description; that map is normalised based on the prevalence of the entire test set to show the prevalence of the files that were used in the test set. As a consequence, the scale on the vendor-specific heat maps and the colours shows are not directly comparable to the test-set description heat map. Commissioned by Microsoft

11 AhnLab V3 Internet Security Czech Republic in Slovak Republic in China in Finland in Netherlands in Belgium in South Korea in Estonia 9975 in Sweden 9814 in Denmark 9781 in France 9651 in Portugal 9595 in Hungary 9560 in Norway 9524 in Brazil 9129 in Slovenia 9089 in Taiwan 8893 in Spain 8871 in Greece 8866 in Iceland 8767 in Global Non-Detection Risk: 5592 in Kishop 2. Bladabindi 3. Dorkbot 4. Lamrek 5. Tesch Commissioned by Microsoft

12 avast! Free Antivirus Ireland 6169 in Estonia 5779 in Uruguay 5749 in Iceland 5713 in France 5658 in Belgium 5026 in Brazil 4834 in Norway 4612 in Germany 4568 in New Zealand 4477 in Denmark 4400 in Finland 4035 in Hong Kong 3778 in Japan 3708 in Bulgaria 3609 in Italy 3588 in South Korea 3453 in Switzerland 3431 in Canada 3419 in Austria 3414 in Global Non-Detection Risk: 1445 in Brantall 2. Necurs 3. Malagent 4. Kuluoz 5. Tesch Commissioned by Microsoft

13 AVG Internet Security Czech Republic 1998 in Slovak Republic 1672 in China 1522 in Iraq 1386 in Ghana 1256 in South Korea 1246 in Portugal 1236 in Israel 1233 in Saudi Arabia 1209 in Tunisia 1198 in Estonia 1163 in Finland 1107 in South Africa 1099 in Taiwan 1083 in Turkey 1081 in Qatar 1079 in Egypt 1050 in Morocco 1033 in Hungary 1032 in Venezuela 1022 in Global Non-Detection Risk: 733 in Balamid 2. Meredrop 3. Bladabindi 4. Dorkbot 5. Necurs Commissioned by Microsoft

14 AVIRA Internet Security Finland 721 in France 638 in Netherlands 624 in Sweden 618 in Czech Republic 596 in Belgium 585 in Denmark 580 in Spain 575 in Norway 530 in Germany 526 in Slovak Republic 514 in Austria 503 in Italy 470 in China 466 in Portugal 446 in Switzerland 440 in Estonia 436 in Greece 426 in Taiwan 425 in Ireland 420 in Global Non-Detection Risk: 198 in Sefnit 2. Dynamer 3. Bladabindi 4. VBInject 5. Kishop Commissioned by Microsoft

15 Baidu Antivirus (en) 1. Ireland 7135 in Uruguay 7105 in France 6562 in Estonia 6119 in Iceland 5864 in Belgium 5843 in Norway 5738 in Brazil 5735 in New Zealand 5547 in Denmark 5289 in Germany 5218 in Chile 4703 in Finland 4531 in Costa Rica 4297 in South Korea 4218 in Bulgaria 4141 in Italy 4119 in Greece 3999 in Jamaica 3883 in Switzerland 3878 in Global Non-Detection Risk: 1890 in Brantall 2. Dorkbot 3. Vundo 4. Kilim 5. Sefnit Commissioned by Microsoft

16 Bitdefender Internet Security Ireland 4959 in Estonia 4751 in Uruguay 4614 in France 4422 in Iceland 4395 in Belgium 4019 in Brazil 3697 in Norway 3685 in Germany 3639 in New Zealand 3559 in Denmark 3392 in Bulgaria 3330 in Finland 3111 in Serbia 2992 in Cyprus 2946 in Greece 2889 in Hungary 2830 in Italy 2806 in Jamaica 2790 in Switzerland 2735 in Global Non-Detection Risk: 1230 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

17 BullGuard Internet Security Ireland 5006 in Estonia 4778 in Uruguay 4742 in France 4461 in Iceland 4412 in Belgium 4063 in Brail 3812 in Norway 3729 in Germany 3665 in New Zealand 3620 in Denmark 3440 in Bulgaria 3377 in Finland 3135 in Serbia 3033 in Cyprus 2990 in Greece 2927 in Hungary 2894 in Jamaica 2850 in Italy 2848 in Tunisia 2822 in Global Non-Detection Risk: 1303 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

18 Emsisoft Anti-Malware Ireland 5005 in Estonia 4778 in Uruguay 4742 in France 4460 in Iceland 4412 in Belgium 4062 in Brazil 3802 in Norway 3728 in Germany 3665 in New Zealand 3619 in Denmark 3439 in Bulgaria 3377 in Finland 3134 in Serbia 3033 in Cyprus 2989 in Greece 2926 in Hungary 2893 in Jamaica 2850 in Italy 2847 in Tunisia 2822 in Global Non-Detection Risk: 1302 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

19 escan Internet Security Ireland 4818 in Uruguay 4620 in France 4319 in Estonia 4268 in Iceland 4206 in Belgium 3839 in Brazil 3701 in New Zealand 3514 in Norway 3512 in Germany 3493 in Denmark 3304 in Finland 2923 in Bulgaria 2760 in Jamaica 2744 in Italy 2718 in Serbia 2698 in Japan 2644 in Switzerland 2628 in Hong Kong 2626 in Cyprus 2625 in Global Non-Detection Risk: 1186 in Brantall 2. Necurs 3. Kishop 4. Gamarue 5. Dorkbot Commissioned by Microsoft

20 ESET Smart Security Hungary 5628 in Slovenia 3615 in Bulgaria 3273 in Estonia 2851 in Ghana 2663 in Turkey 2559 in Tunisia 2293 in Slovak Republic 2244 in South Korea 2120 in Greece 2096 in China 2004 in Lithuania 1899 in Netherlands 1853 in Cyprus 1703 in Portugal 1681 in Serbia 1640 in Romania 1568 in Belgium 1544 in Poland 1539 in Finland 1480 in Global Non-Detection Risk: 641 in Kilim 2. Malagent 3. Sisron 4. Dynamer 5. Bladabindi Commissioned by Microsoft

21 F-Secure Internet Security Ireland 3524 in Uruguay 3388 in France 3168 in Estonia 3158 in Iceland 3099 in Belgium 2799 in Brazil 2726 in Norway 2560 in New Zealand 2555 in Germany 2528 in Denmark 2412 in Finland 2142 in Bulgaria 2025 in Italy 1976 in Japan 1973 in Chile 1942 in Hong Kong 1936 in Jamaica 1928 in Serbia 1921 in Switzerland 1883 in Global Non-Detection Risk: 821 in Brantall 2. Necurs 3. Kishop 4. Dorkbot 5. Bladabindi Commissioned by Microsoft

22 Fortinet FortiClient France 3025 in Ireland 2783 in Belgium 2581 in Finland 2408 in Uruguay 2370 in Denmark 2364 in Germany 2350 in Norway 2296 in Estonia 2286 in Iceland 2273 in Brazil 2095 in New Zealand 2051 in Spain 1921 in Sweden 1893 in Austria 1858 in Italy 1806 in Switzerland 1772 in Netherlands 1754 in Japan 1738 in Greece 1679 in Global Non-Detection Risk: 572 in Brantall 2. Sefnit 3. Notorgatro 4. Necurs 5. Bancos Commissioned by Microsoft

23 Kaspersky Internet Security (e) 1. China 681 in South Korea 602 in United States 504 in Canada 407 in Taiwan 334 in Estonia 329 in Czech Republic 314 in Hong Kong 297 in Brazil 294 in Latvia 288 in Slovak Republic 284 in Finland 271 in Sweden 255 in Netherlands 254 in Iceland 244 in Slovenia 243 in Spain 241 in Hungary 237 in Portugal 233 in Lithuania 226 in Global Non-Detection Risk: 121 in Tesch 2. Dynamer 3. Necurs 4. Sodebral 5. Ramnit Commissioned by Microsoft

24 Kingsoft Internet Security 2013.SP Ireland 5005 in Estonia 4777 in Uruguay 4737 in France 4457 in Iceland 4411 in Belgium 4059 in Brazil 3804 in Norway 3727 in Germany 3660 in New Zealand 3618 in Denmark 3438 in Bulgaria 3375 in Finland 3134 in Serbia 3032 in Cyprus 2986 in Greece 2925 in Hungary 2892 in Jamaica 2849 in Italy 2843 in Tunisia 2789 in Global Non-Detection Risk: 1294 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

25 Lavasoft Ad-Aware Free Antivirus Ireland 5005 in Estonia 4776 in Uruguay 4737 in France 4457 in Iceland 4411 in Belgium 4059 in Brazil 3804 in Norway 3727 in Germany 3660 in New Zealand 3618 in Denmark 3437 in Bulgaria 3375 in Finland 3133 in Serbia 3031 in Cyprus 2986 in Greece 2925 in Hungary 2892 in Jamaica 2849 in Italy 2843 in Tunisia 2789 in Global Non-Detection Risk: 1293 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

26 McAfee Internet Security China 928 in Brazil 823 in South Korea 552 in Taiwan 452 in Portugal 373 in Czech Republic 285 in Italy 277 in Netherlands 275 in Estonia 273 in Slovak Republic 269 in Hong Kong 254 in Belgium 249 in Finland 245 in Denmark 236 in Latvia 230 in Sweden 229 in Norway 228 in Egypt 226 in France 223 in Germany 216 in Global Non-Detection Risk: 122 in Sulunch 2. Banker 3. Dynamer 4. Sisron 5. Bancos Commissioned by Microsoft

27 Microsoft Security Essentials Czech Republic 1026 in Slovak Republic 865 in South Korea 857 in China 776 in Netherlands 723 in Sweden 674 in Finland 654 in Denmark 645 in Croatia 629 in Portugal 624 in Brazil 624 in Estonia 620 in Iraq 620 in Norway 617 in Belgium 617 in Saudi Arabia 603 in Latvia 600 in Iceland 597 in Austria 595 in Taiwan 590 in Global Non-Detection Risk: 337 in CeeInject 2. Stekct 3. Bladabindi 4. Dynamer 5. Ainslot Commissioned by Microsoft

28 Panda Cloud Free Antivirus Ireland 1037 in Uruguay 970 in Estonia 951 in France 945 in Iceland 907 in Brazil 879 in Belgium 851 in Norway 788 in New Zealand 759 in Germany 754 in Denmark 745 in Finland 687 in South Korea 659 in Italy 598 in Hong Kong 594 in Bulgaria 585 in Japan 574 in Serbia 559 in Switzerland 559 in Greece 557 in Global Non-Detection Risk: 225 in Brantall 2. SystemHijack 3. Dynamer 4. Necurs 5. Dofoil Commissioned by Microsoft

29 Qihoo 360 Internet Security (en) 1. Ireland 5006 in Estonia 4778 in Uruguay 4742 in France 4461 in Iceland 4412 in Belgium 4063 in Brazil 3808 in Norway 3729 in Germany 3665 in New Zealand 3620 in Denmark 3440 in Bulgaria 3377 in Finland 3135 in Serbia 3033 in Cyprus 2990 in Greece 2927 in Hungary 2894 in Jamaica 2850 in Italy 2848 in Tunisia 2822 in Global Non-Detection Risk: 1303 in Brantall 2. Kilim 3. Necurs 4. Gamarue 5. Kishop Commissioned by Microsoft

30 Sophos Endpoint Security and Control China 1897 in South Korea 1507 in Hungary 1495 in Estonia 1396 in Slovenia 1260 in Czech Republic 1210 in Greece 1205 in Slovak Republic 1200 in Bulgaria 1065 in Hong Kong 1054 in Lithuania 1049 in Finland 1016 in Latvia 999 in Taiwan 999 in Netherlands 954 in Brazil 954 in Iceland 938 in Sweden 910 in Turkey 884 in Portugal 884 in Global Non-Detection Risk: 594 in Kilim 2. Dynamer 3. Vundo 4. Necurs 5. Tesch Commissioned by Microsoft

31 Tencent QQ PC Manager Finland 809 in Sweden 704 in Netherlands 701 in Czech Republic 693 in France 690 in China 675 in Belgium 651 in Denmark 649 in Spain 632 in Norway 600 in Slovak Republic 590 in Germany 574 in Austria 550 in Estonia 522 in Italy 514 in Taiwan 510 in South Korea 502 in Portugal 499 in Hungary 465 in Ireland 464 in Global Non-Detection Risk: 200 in Sefnit 2. Dynamer 3. VBInject 4. Bladabindi 5. Kishop Commissioned by Microsoft

32 ThreatTrack Vipre Internet Security Japan 981 in Ghana 771 in Canada 761 in Hong Kong 741 in United Kingdom 738 in Austria 733 in Croatia 720 in Australia 707 in Italy 698 in Switzerland 694 in Pakistan 679 in Romania 658 in Germany 655 in Albania 641 in Czech Republic 640 in Egypt 633 in Netherlands 627 in Iceland 623 in Finland 621 in Latvia 619 in Global Non-Detection Risk: 528 in Zbot 2. Gamarue 3. VBInject 4. Necurs 5. Malagent Commissioned by Microsoft

33 Trend Micro Titanium Internet Security Ireland 5719 in Uruguay 5441 in France 5054 in Estonia 4843 in Iceland 4731 in Belgium 4511 in Brazil 4343 in New Zealand 4202 in Norway 4161 in Germany 4101 in Denmark 3941 in Finland 3428 in Italy 3144 in Bulgaria 3084 in Chile 3080 in Japan 3025 in Jamaica 3024 in Switzerland 2998 in Serbia 2975 in Canada 2941 in Global Non-Detection Risk: 1181 in Brantall 2. Dorkbot 3. Notorgatro 4. Zbot 5. Sirefef Commissioned by Microsoft

34 Copyright and Disclaimer This publication is Copyright 2014 by AV-Comparatives. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV- Comparatives, prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV- Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data. For more information about AV-Comparatives and the testing methodologies, please visit our website. AV-Comparatives (August 2014) Commissioned by Microsoft