AVG PC TuneUp 2014 Perfomance Test Whitepaper

Transcription

1 AVG 14 Perfomance Test Whitepaper Date Serving 17 million active users. Whatever your device, we ve got you covered. Version 1.4

2 Contents 1 Introduction... 1 Test Environment & Guidelines Test Tools... 3 Boot Time & Shutdown Tests Preparing the test machines Test Machine Specifications...5 Installing 15+ applications Boot Results era Ultrabook era Microsoft Surface Pro Shutdown Results File Copying Results Photo Editing Results Background Processes Results Memory Usage Test ing Test era Ultrabook era Microsoft Surface Pro era Ultrabook era Microsoft Surface Pro era Acer Iconia W era Ultrabook era Microsoft Surface Pro era Ultrabook era Microsoft Surface Pro era Acer Iconia W era Ultrabook era Microsoft Surface Pro era Acer Iconia W ing Results 14GB Average Space Gained 3 AVG 14 Date Version 1.4

3 1 Introduction This document is meant to protocol performance tests for AVG PC TuneUp. It is important to know that these results were performed on sample machines and do not necessarily represent all laptops. However both low-powered and high-powered devices were tested in order to give an accurate indication of potential results across a range of machines that are currently available on the market. AVG 14 Date Version 1.4 1

4 Test Environment & Guidelines 1 The tested devices resided in an environment compliant with ECMA-383: Temperature: 3 degrees Celsius Relative humidity: 1-8 % Ambient light: 5 +/- 5 lux The Power Plan has been set to Balanced 3 All external devices have been unplugged (keyboard, mouse, hard disks ) 4 Wireless adapters have deliberately been enabled to recreate a typical user scenario 5 All tests were performed according to Microsoft Developer Central Hardware guidelines: All machines have been used productively for 4 weeks The Rundll3.exe advapi3.dll,processidletasks command was used by internal testers to make Windows perform file placement optimization tasks Windows Update was successfully run on all machines 4 hours prior to testing and the systems were each restarted five times. Windows Update was then disabled The Windows Experience Index was calculated Windows 8 s new Automatic Maintenance feature was first executed and then disabled The SuperFetch feature, which adapts to programs and Windows usage, was left enabled The Windows Search index was fully built Scheduled Tasks have been performed 6 We ran each benchmark test three times and used the average in our results 7 All the devices were fully charged to 1% and ran down until 1% AVG 14 Date Version 1.4

5 3 Test Tools Testers had to perform several steps and follow a specific flow. In order to get accurate results, we used both an automated and a manual approach. 3.1 Boot Time & Shutdown Tests For boot time tests, we used Microsoft s Windows Performance Toolkit, part of the Windows Assessment Toolkit: The assessment results are used to diagnose potential problems, so that the hardware and software that you develop are both responsive and have a minimal impact on battery life, startup performance, and shutdown time. The same assessments are available for OEM partners, ISV/IHV partners, enthusiasts, and other members of the community, to establish a common framework to measure, compare, and review aspects of quality. This kit performs extensive boot traces and covers the entire boot process, from UEFI/BIOS initialization until the last service is run. It gives an indication of the impact of 3rd party software on the boot process. On both tablets, testers used the Fast Startup check which measures the Hybrid Boot feature found on Windows 8. On the ultrabook, testers were forced to perform a full boot which is similar to the boot experience found on Windows 7 and earlier versions. AVG 14 Date Version 1.4 3

6 3 Test Tools 3.1 Boot Time & Shutdown Tests cont... For shut down tests, Windows Performance Toolkit was used to measure the PC shutdown duration down to the millisecond. For resource usage tests (number of processes and memory usage), Task-Manager was used. In order to eliminate as much post-boot interference as possible, the PC was first booted and the values were then taken 15 minutes after the boot procedure had completed. For photo editing benchmarks, the Photo Handling assessment from Microsoft Assessment Toolkit was used. This measures the time it takes to complete user tasks such as viewing, searching and manipulating a set of photos. For file operation tasks, the File Handling assessment from Microsoft Assessment Toolkit was used. This measures the duration of common file functions such as copy, move, delete, and zip. AVG 14 Date Version 1.4 4

7 4 Preparing the test machines 4.1 Test Machine Specifications Benchmarks were performed on recent machines (Q4 11-Q 13) with Windows 8, which is the most performance-optimized Windows OS to date. 11-era Ultrabook Core i7, 4 GB, 56 GB (Purchased: September 11, Battery health was rated at 9%) 1-era Samsung Series 7 Tablet Core i5 1.6 GHz, 4 GB, 64 GB SSD (Purchased: May 1) 13-era Microsoft Surface Pro Core i5 1.7 GHz, 4 GB, 18 GB SSD (Purchased: June 13) It should be noted that the results are likely to be even greater on less performance-oriented operating systems, such as Windows XP or Windows Vista. AVG 14 Date Version 1.4 5

8 4 Preparing the test machines 4. Installing 15+ applications With every installed program, there s increased load on the system. This load consists of: Additional background activity puts strain on a user s machine, as it needs to assign a certain amount of resources to the newly installed programs, which reduces performance. Two factors are responsible for this loss in performance: The system becomes slower with each program that gets installed. In this experiment, a large (but not unusual) amount of applications was installed to see the effect on performance. Services Background processes Startup items Plug-Ins and Explorer.exe hooks Drivers (some software products even install new drivers, for example virtual DVD drives.) Windows needs to assign resources (CPU time, handles) to these background tasks Less memory is available for active processes AVG 14 Date Version 1.4 6

9 4 Preparing the test machines 4. Installing 15+ applications cont... This is a typical scenario to test the effect of AVG on systems with a high load. The experiment helps evaluate how a high application load has an effect on modern systems and whether AVG is capable of optimizing performance. The following steps were performed: 1 The three test machines used were deliberately left in an untouched condition to simulate three typical user scenarios: Ultrabook: An ultrabook was clean-installed in January of 13 and few programs installed. It is an example of a relatively newly installed machine with little load. Microsoft Surface Pro: The Surface Pro was a clean installed and had no programs on it. The testers only needed to install benchmarking software and the appropriate tools to determine the test results. Acer Iconia W3: This machine was pre-installed by the OEM and came with 15 built-in applications and trial versions. Testers prepared the test machines according to Microsoft guidelines (see above). 3 An image of the clean install was taken using Microsoft s built-in recimg. The Windows 8 Refresh features could be used to revert to this clean state. All three PCs in their untouched condition are referred to as. AVG 14 Date Version 1.4 7

10 4 Preparing the test machines 4. Installing 15+ applications cont... 4 To simulate workload, the machines were equipped with approximately 15 additional popular programs. This high number of applications is not an uncommon scenario whether the user is a beginner (who may not understand the effect on performance) or an IT pro (who requires a significant amount of applications). Identifying programs: Testers selected these 15 based on the most used programs on Wakoopa (a software usage tracker), the top downloads at Downloads.com, from various Top 1 Programs you need lists and from personal experience of what users often install. The popular Ninite installer was also used: This tool silently installs over 5 popular applications (double software was unchecked). Overall, the programs installed as part of this test represent an even mixture of very popular and huge software suites such as Microsoft Office or Nero and freeware applications such as Skype and Opera. Installation process: The programs were installed using their Standard installation options, which in many cases also installed additional toolbars or downloads. This causes an additional load on the machine. 5 After the installation was completed, all programs were started at least once on all three test devices to determine their functionality or set them up. 6 Each of the three systems was then rebooted 15 times over the course of 3 consecutive days. This removes the effect of most of the post-installation background activities that applications frequently perform. At this point, the state of these systems is referred to as High Load. An image tool was then used to perform complete system images. 7 All benchmarks below were performed according to the Microsoft guidelines. AVG 14 Date Version 1.4 8

11 4 Preparing the test machines 4. Installing 15+ applications cont... 8 AVG was installed on these devices and the following performance-optimizing steps were performed: 1-Click-Maintenance: Run with default settings Increase Performance Assistant: Profile set and all recommendations accepted Fix Problem Assistant: All problems were fixed Turbo Mode: Enabled Program Deactivator: All programs were turned off (two reboots were necessary) Startup Manager: All remaining startup entries were turned off Live Optimization: Enabled Registry Defrag: Performed This optimized state is referred to as Optimized with AVG Disclaimer The following test results were performed under a highly controlled environment, in accordance to industry standards, and with professional measurement software which was previously only used by Microsoft engineers (Windows Performance Toolkit was an internal Microsoft tool to measure OS performance). The tests were done with care and repeated several times. However, the testers cannot guarantee that these performance tests can be reproduced on other machines. While performance testing on clean PCs is straightforward, the installation of several programs introduces factors that cannot be controlled these include sudden interference by update mechanisms or self-maintenance tasks that the installed programs perform after a certain time or when triggered. These variances were reduced to a minimum by several reboots and days of uptime however, they cannot be eliminated. It is possible that the results were impacted due to the high load that was put on the system. Nevertheless, the results represent a strong and accurate representation of systems under high load and how an optimization product is capable of solving performance problems. AVG 14 Date Version 1.4 9

12 5 Boot Results Boot time cannot be measured subjectively (e.g. stop watch technique), as Windows performs a number of post-boot operations that have a severe impact on performance. Only when these operations are done, can users fully enjoy their PC s performance. Windows Performance Toolkit traces the entire boot procedure from UEFI/BIOS boot until the last service has loaded and gives an accurate representation of boot time. AVG 14 Date Version 1.4 1

13 5 Boot Results Windows Boot Time Tests era Ultrabook: 63% Faster Bootup Faster Boot Up on Ultrabook 6 sec 164 min High Load (15+ Programs) 6 sec SECONDS The ultrabook saw problems with the Hybrid Boot feature of Windows 8, which is why the Full Boot test suite was used. This performs a regular boot, which is similar to the procedure found in older Windows versions. In our tests, the ultrabook s boot procedure was completed after 6 seconds. However, Faster Boot Up oninstalled, Microsoft Surface Proseconds. After the once the 15 programs were boot time went to 164 optimization with AVG, the values didn t just return to normal but were even lower (6 seconds) than on the clean system. This is a reduction in boot time of 63.41%. 7. sec Average High Load Optimized with AVG 6 seconds 164 seconds 6 seconds 57.8 sec High Load (15+ Programs) *Windows Performance Toolkit performs 3 test runs in total and averages the results. All test details and test dumps are available. 7.8 sec SECONDS AVG 14 Date Version

14 Faster Boot Up on Ultrabook 6 sec 164 min High Load (15+ Programs) 6 sec 5 Boot Results SECONDS 5. 1-era Microsoft Surface Pro: 87% Faster Bootup Faster Boot Up on Microsoft Surface Pro 7. sec 57.8 sec High Load (15+ Programs) 7.8 sec SECONDS Total boot time went from 57.8 seconds (after the programs were installed) down to 7.8 seconds which is almost as low as when the Surface Pro was tested in its clean condition (7. seconds). In short, turning off the entire additional load on the system and enabling all performance-related features of AVG, the Microsoft Surface Pro saw a decrease in boot time of 86.51%. Average High Load Optimized with AVG 7. seconds 57.8 seconds 7.8 seconds *Windows Performance Toolkit performs 3 test runs in total and averages the results. AVG 14 Date Version 1.4 1

15 6 Shutdown Results Restarting and shutting down the system is a valid indicator of how fast the PC is capable of ending processes and services. Also, if a shutdown takes too long, users tend to get frustrated a shorter shutdown means a generally better user experience. AVG 14 Date Version

16 6 Shutdown Results Windows Shutdown Time Tests era Ultrabook: 9% Faster Shutdown Faster Shutdown on Ultrabook 3.7 sec 18.9 sec High Load (15+ Programs) 1.8 sec SECONDS The time it took to shut down the ultrabook went from 3.7 seconds to 18.9 once all the programs were installed. AVG was able to optimize this PC significantly: After the optimization, it only took 1.8 seconds to shut down the PC. This is a reduction of boot time of 9.47%! Faster Shutdown on Microsoft Surface Pro 4.5 sec *Average High Load Optimized with AVG 3.7 seconds 18.9 seconds 1.8 seconds 11.3 sec High Load (15+ Programs) *Windows Performance Toolkit performs 3 test runs in total and averages the results. 6 sec 4 AVG SECONDS 1 Date Version

17 Faster Shutdown on Ultrabook 3.7 sec 18.9 sec High Load (15+ Programs) 1.8 sec 6 Shutdown Results SECONDS 6. 1-era Microsoft Surface Pro: 47% Faster Shutdown Faster Shutdown on Microsoft Surface Pro 4.5 sec 11.3 sec High Load (15+ Programs) 6 sec SECONDS The shutdown procedure took slightly more than 11 seconds after all the programs were installed on the Surface Pro. However, once AVG performed all tuning operations, this value decreased noticeably from 11 seconds to 6. This is an improvement of 46.9%. *Average High Load Optimized with AVG 4.5 seconds 11.3 seconds 6. seconds *Windows Performance Toolkit performs 3 test runs in total and averages the results. AVG 14 Date Version

18 7 File Copying Results File I/O can be severely impacted by background processes consuming both hard disk cycles as well as CPU time. Windows Assessment Toolkit performed large and small file operations repeatedly to determine the hard disk performance. AVG 14 Date Version

19 7 File Copying Results 7.1 File 11-era Ultrabook: 9% Faster File Copy Copying Results Faster File Copy on Ultrabook 8.7 sec 9.5 sec High Load (15+ Programs) 8.6 sec SECONDS File copy operations on the ultrabook were only slightly impacted (9.5 versus 8.7 seconds) by the additional load, which is due to the stronger Core i7 processor and the SSD. The optimization with AVG returned this value back to normal. This is an improvement 9.47%. FasterofFile Copy on Surface Pro 5.6 sec High6.6 Load sec High Load (15+ Programs) 8.7 seconds *Average Optimized with AVG 9.5 seconds 8.6 seconds 5.8 sec *Windows Performance Toolkit performs 3 test runs in total and averages the results SECONDS AVG 14 Faster File Copy on Acer Iconia W3 Date Version

20 File Copying Results Faster File Copy on Ultrabook 8.7 sec 9.5 sec High Load (15+ Programs) 7 File Copying Results sec SECONDS 7. 1-era Microsoft Surface Pro: 1% Faster File Copy Faster File Copy on Surface Pro 5.6 sec 6.6 sec High Load (15+ Programs) sec SECONDS The additional load of 15 programs had a slight impact on copy operations (6.6 versus 5.6 seconds), which testers noted in several repetitions of the benchmarks. The optimization with AVG improved the file copy results, but could not revert it back to the originalfaster value. Still, it is an improvement of 1.1%. File Copy on Acer Iconia W3 7 sec High Load (15+ Programs) Optimized with AVG 6.6 seconds 5.8 seconds 15 sec 5.6 seconds *Average High Load 64 3sec *Windows Performance Toolkit performs test runs in total and averages the results SECONDS AVG 14 Date Version

21 Faster File Copy on Surface Pro 5.6 sec 6.6 sec High Load (15+ Programs) 7 File Copying Results sec SECONDS era Acer Iconia W3: 58% Faster File Copy Faster File Copy on Acer Iconia W3 7 sec 15 sec High Load (15+ Programs) 64 sec SECONDS The low-end processor, also commonly found in netbooks, was not designed to handle both intense background processes and CPU-intense file copy operations. It took the device 15 seconds to complete the task. After the optimization, it took the device only 64 seconds to complete the task. This is an improvement of 57.89%. *Average High Load Optimized with AVG 7 seconds 15 seconds 64 seconds *Windows Performance Toolkit performs 3 test runs in total and averages the results. AVG 14 Date Version

22 8 Photo Editing Results This test measures the performance when working with photos (small to large). Tasks that are measured several times are: Opening a large picture folder, building the thumbnail index, opening photos, rotating pictures and manipulating pictures. The results are not shown in seconds. Instead, Microsoft Assessment tools trace delays that are visible to the user. If there is no visible delay in the operation on screen (e.g. opening a picture), the value of operations without delay is 1%. However, if there is visible impact, the percentage is lower. For example, if the value is 9%, it means that in 8% there was a noticeable lag in editing or viewing pictures. AVG 14 Date Version 1.4

23 8 Photo Editing Results 8.1 Photo 11-era Ultrabook: 4% Fewer Delays Editing Results Photo Editing Improvements on Ultrabook 96% 9% High Load (15+ Programs) 96% PERCENT On the clean ultrabook, the assessment tool detected delays in only about 4% of all photo- related tasks. After the system was under the load of 15+ programs, this value increased to 8%. AVG was able to restore performance to the original level, which meansphoto there were 4% less delays. Editing Improvements on Surface Pro 1% High Load 1% High Load (15+ Programs) 96% *Average 9% Optimized with AVG 96% 1% *Windows Performance Toolkit performs 3 test runs in total and averages the results PERCENT AVG 14 Date Version 1.4 1

24 Photo Editing Results Photo Editing Improvements on Ultrabook 96% 9% High Load (15+ Programs) Results 8 Photo Editing % 8 1 PERCENT 8. 1-era Microsoft Surface Pro: % Fewer Delays Photo Editing Improvements on Surface Pro 1% High Load (15+ Programs) 1% 1% PERCENT The Surface Pro was the only device that did not suffer under the load of multiple applications being run in the background. The photo performance did not change. Resonse without delays High Load Optimized with AVG 1% 1% 1% *Windows Performance Toolkit performs 3 test runs in total and averages the results. AVG 14 Date Version 1.4

25 9 Background Processes Results Day-to-day performance and responsiveness is determined by the amount of active processes as well as free memory. This test measures the number of active processes before and after optimizing the test devices with AVG. AVG 14 Date Version 1.4 3

26 9 Background Processes Results 9.1 Resource 11-era Ultrabook: 7%Results Fewer Active Processes Usage Less Active Processes Ultrabook High Load (15+ Programs) NUMBER OF PROCESSES The number of active processes jumped from 69 to 148 and reduced the responsiveness drastically. The task-manager showed a constant activity of 1-3% (CPU usage). After the optimization with was complete, there were only 4 processes active Less Active Processes Surface Pro a reduction of 71.6%. 4 # of background processes High Load Optimized with AVG Test Run 1 69 processes 146 processes 4 processes Test Run 69 processes High Load (15+ Programs) processes 4 processes Test Run 3 68 processes 148 processes 4 processes 69 processes 148 processes 4 processes Average NUMBER OF PROCESSES AVG 14 Less Active Processes Acer Iconia W3 Date Version 1.4 4

27 Resource Usage Results Less Active Processes Ultrabook High Load (15+ Programs) 4 Processes Results 9 Background NUMBER OF PROCESSES 9. 1-era Microsoft Surface Pro: 66% Fewer Active Processes Less Active Processes Surface Pro High Load (15+ Programs) NUMBER OF PROCESSES On the clean Microsoft Surface Pro, only 4 processes were active as it only had Windows 8 and few other programs installed. However, after the 15+ products were installed and the system rebooted 15 times and used Iconia for severalw3 days, this number jumped to Lesswas Active Processes Acer 133. While AVG improved the situation (45 processes, a reduction of 66.16%), it wasn t quite able to get it back into its original shape. 46 # of background processes High Load Optimized with AVG Test Run 1 4 processes17 High Load (15+ Programs) 133 processes 45 processes Test Run 4 processes 13 processes 45 processes 4 processes 133 processes 45 processes Test Run 3 Average processes 4 processes processes 175 NUMBER OF PROCESSES AVG 14 Date Version 1.4 5

28 Less Active Processes Surface Pro High Load (15+ Programs) 9 Background 45 Processes Results OF PROCESSES era Acer IconiaNUMBER W3: 57% Fewer Active Processes Less Active Processes Acer Iconia W3 46 High Load (15+ Programs) NUMBER OF PROCESSES After the additional programs were installed, there were 17 active processes putting heavy load on the low-powered Dual-Core CPU. It was no longer possible to perform work, launch apps, or even browse the web without significant delays. After optimization, only 46 processes were active and the system responded a lot faster this is a reduction of processes by 57.1%. # of background processes High Load Optimized with AVG Test Run 1 51 processes 17 processes 46 processes Test Run 5 processes 17 processes 46 processes Test Run 3 51 processes 18 processes 46 processes Average 51 processes 17 processes 46 processes AVG 14 Date Version 1.4 6

29 1 Memory Usage Test To complete the picture of PC responsiveness, we need to examine the amount of free memory. To this end testers measured how much memory was available before and after optimizing the test devices with AVG. AVG 14 Date Version 1.4 7

30 1 Memory Usage Test era Ultrabook: 7% (1.6GB) Memory Saved 1.6GB Memory Saved Ultrabook High Load (15+ Programs) GIGABYTES The amount of memory used on this 4 GB system increased from 1.1 Gigabytes to.3. This means less available resources for more actively used programs. AVG s optimization saved 1.6 GB of RAM (a saving of 69.57%) 1.1GB Memory Saved Surface Pro.9 Memory Usage (Committed) High Load Test Run GB High Load (15+ Programs).4 GB Test Run 1.1 GB.4 GB.8 GB Test Run GB.4 GB.8 GB.4 GB.8 GB GB Average Optimized with AVG GB..5 GIGABYTES AVG 14.8GB Memory Saved Acer Iconia W3 Date Version 1.4 8

31 1.6GB Memory Saved Ultrabook High Load (15+ Programs).8 Test 1 Memory Usage GIGABYTES 1. 1-era Microsoft Surface Pro: 5% (1.1GB) Memory Saved 1.1GB Memory Saved Surface Pro.9. High Load (15+ Programs) GIGABYTES The amount of memory consumed by the OS, drivers and 3rd party applications rose from.9 to. GB. and its various optimization tools were able to reduce this number drastically, saving 1.1 GB (a saving of 5%)..8GB Memory Saved Acer Iconia W3.7 Memory Usage (Committed) High Load Test Run 1.9 GB. GB Test Run.9 GB. GB Test Run 3.9 GB. GB 1.1 GB.6.9 GB. GB 1.1 GB High Load (15+ Programs) Average.5 1 Optimized with AVG 1.1 GB GB 1.5 GIGABYTES AVG 14 Date Version 1.4 9

32 1.1GB Memory Saved Surface Pro.9. High Load (15+ Programs) TuneUp MemoryPCUsage Test GIGABYTES era Acer Iconia W3: 5% (.8 GB) Memory Saved.8GB Memory Saved Acer Iconia W High Load (15+ Programs) GIGABYTES As the Aconia W3 tablet had only GB of RAM, having 1.4 GB completely eaten up after the installation of over 15 applications resulted in a massive decrease in performance. AVG, however, saved 8 Megabytes of memory on this tablet (a saving of 5%). Memory Usage (Committed) High Load (18 Programs) Optimized with AVG Test Run 1.7 GB 1.4 GB.6 GB Test Run.7 GB 1.4 GB.6 GB Test Run 3.7 GB 1.4 GB.6 GB Average.7 GB 1.4 GB.6 GB AVG 14 Date Version 1.4 3

33 11 ing Test The following cleaning tests (using Disk er) were performed after the 15 programs were installed. It should be noted that the following values represent snapshots only and could change as the system is used. AVG 14 Date Version

34 11 ing Test 11.1 ing Results 14GB Average Space Gained Core i7 Ultrabook Microsoft Surface Pro Acer Iconia W3 MacBook Pro Retina (running Windows) 3.8 GB s of 6.8 GB s of unnecessary files unnecessary files wasteful wasteful wasteful wasteful files in total files in total files in total files in total 8.5 GB s of unnecessary files GB s of unnecessary files On average, this equates to 14 Gigabytes of gained disk space. For users, 14 Gigabytes equates to roughly: 67 More Photos 5 More Songs 5 hrs more HD Video (average size:.1mb) (average size:.1mb) (average size:.1mb) Legal Disclaimer Wi-Fi is a registered trademark of Wi-Fi Alliance. Bluetooth is a registered trademark of Blueooth SIG. Samsung and Samsung Series 7 are registered trademarks of Samsung Electronics Co. Windows, Windows XP, Windows Vista, Windows 7, Windows 8, Surface Pro and Internet Explorer, are registered trademarks of Microsoft Corporation. PowerMark is a registered trademark of Futuremark Corp. AVG 14 Date Version 1.4 3