White Paper File and Networking Services Performance Test Report: Novell iprint Appliance. Table of Contents page Executive Summary.... Introduction.... Overview... 3. Configurable Test Variables...3 4. Test Methodology...4 5. Test Lab Design...4 6. Tested Scenarios...5 7. Comparisons...7 8. Conclusions...9
File and Networking Services White Paper Performance Test Report: Novell iprint Appliance. Executive Summary This report presents Novell iprint Appliance job-rendering performance results that Novell testers observed while conducting tests to capture and illustrate the relationships between typical print-traffic scenarios and iprint Appliance ecosystems. Several variables can contribute to print traffic, including job volumes, specific formats and submission rates. The iprint Appliance ecosystem contains multiple Windows renderers, including single- and dual-core based Windows renderers. The primary relationships between print traffic and iprint Appliance are job-rendering and job-wait times. To establish these relationships, testers used the iprint server s built-in audit log to track job submissions, rendering times, formats and sizes. This document contains information about the test environment, the tests and the test results. NOTE: Novell iprint exists as both a part of Novell Open Enterprise Server and as a stand-alone software appliance. The tests in this document are applicable only to Novell iprint Appliance... Introduction This document provides information about performance results from several tests Novell conducted. The report focuses on the times it took to print jobs that a mobile device submitted to a printer. In each test, iprint Appliance receives a print job from a mobile client and sends it to one or more internal or external renderers to convert the job to a print-ready state. Testers considered several parameters for running these tests: Print job size Print job format Internal rendering rendering Multiple external renderings CPU- and file I/O-intensive security-compliance applications such as antivirus software Testers incorporated antivirus software during testing and used the built-in iprint auditing function to derive test results. The results in this report do not indicate internal or external renderers maximum performance capabilities. Novell conducted the tests to achieve the following goals: Understand rendering performance in different job submission scenarios Arrive at the optimal number of Windows renderers required for different job loads. Overview Novell iprint eliminates complex printing environments. It is available as a virtual appliance that offers cross-platform, self-service printing for the enterprise. Novell iprint allows users to send documents to any printer via a web browser from their desktop, laptop or mobile device. It also integrates with your Active Directory or edirectory environment to provide seamless authentication for your users. Novell iprint can save your IT department time and money by significantly reducing the management burden normally required for mixed fleets of Windows, Mac and Linux desktops. It works across the major
mobile platforms as well, meaning IT only has to manage a single print solution for all of your organization s endpoints. It works with your current users in Active Directory or edirectory to provide authentication. Novell iprint scales exceptionally well, which reduces the server infrastructure you need to maintain hundreds or thousands of printers, thus lowering your network bandwidth costs. You can mobilize your existing fleet of printers rather than buying new ones and you can implement print monitoring to track printing costs. With user selfprovisioning of network printers, you can reduce helpdesk expenses because of fewer calls for printer help. Using a web browser, users working at desktops or laptops can choose printers on a graphical office map. The map shows the location and details of every printer in the company (both local and global). Users can begin printing with just a few mouse clicks no training required. Users on Windows, Mac and Linux can send print jobs through iprint to both new and legacy printers from all the major print vendors, without worrying about print drivers. Mobile Printing Printing normally requires drivers to convert documents into printerready languages. Some vendors use cloud services to convert documents from mobile devices, but iprint Appliance does not require you to send documents through a cloud service and does not require any specific hardware. By keeping information out of the cloud, iprint Appliance is more secure. It also allows users to use their current infrastructures and avoids vendor lock in. Novell iprint Appliance contains its own rendering and conversion capabilities, which allow it to receive print jobs from a variety of mobile sources. For more information, see the Novell iprint Appliance Administration Guide at: www.novell.com/documentation/iprint-appliance/ 3. Configurable Test Variables Testers used the following parameters and conditions for the tests: Job Size: Testers considered file sizes ranging from a few KBs to MBs. They tested on both external and internal renderers and incorporated various file sizes into the test matrix to understand typical wait times for users submitting these jobs. Job Format: Testers used different job formats to study time-takento-render performance with internal and external renderers. Internal Rendering: iprint Appliance is bundled with an internal document renderer. The internal renderer converts documents to the PDF format, which is then translated into a print-ready format using the built-in open source printing system. The renderer supports Open Office, Microsoft Office and image formats, and it also supports multi-threaded rendering to handle heavy loads. For more information about the file types the renderer supports, see the Novell iprint Appliance Administration Guide at: www.novell.com/ documentation/iprint-appliance/ Rendering: You can only configure the option to use an external renderer on a Windows system that has additional software such as Adobe Acrobat X Pro, Microsoft Office, or an iprint rendering client registered to iprint Appliance. Because this setup contains two different applications, testers tested the supported formats for both applications. Adobe supports Portable Document Format (PDF) and Microsoft Office supports several proprietary and open source formats. For more information about iprint Appliance formats for Windows Office, see the Novell iprint Appliance Administration Guide at: www.novell.com/documentation/iprint-appliance/ Multiple Rendering: iprint Appliance supports multiple external renderers, which means your administrators can register one or more Windows instances to iprint Appliance. This type of iprint Appliance configuration does not incur additional format support. The rendering time improves with the addition of each external renderer, thereby reducing the time taken to print. For more information on performance improvements with additional renderers, see section 6 Tested Scenarios. Antivirus Applications: You install iprint renderers on Windows machines. Enterprises usually have several applications installed on systems for security compliance antivirus (AV) solutions, for example. AV scans are processor intensive, so testers ran tests on iprint Windows renderers with AV solutions installed and running. Job Submission Rates: Normal Load: Testers created and submitted jobs of different volumes for a period of one hour ( minutes) jobs per hour and 5 jobs per hour, for example. They emulated this job traffic from the current page per minute (PPM) handling capability of enterprise-class printers, defining these loads as normal loads. Peak Load: By definition, peak-load traffic occurs over a short duration during which job-submission rates are very high, thereby stressing the printing subsystem and increasing users wait times. www.novell.com 3
File and Networking Services White Paper Performance Test Report: Novell iprint Appliance. The purpose of this test was to capture the wait time for the last job across single to multiple renderers. Testers tested peak loads of jobs. 4. Test Methodology Testers primary objective was to show the results of different appliance setups pertaining to different factors such as multiple renderers and hardware requirements that would help improve job wait times. Areas outside the test scope included mail-arrival latency and network bandwidth between mobile devices and the iprint Appliance server. Rather than include these areas in their job-submission workflows, testers identified areas over which iprint Appliance has control in the submission process to uncover potential bottlenecks and understand the impacts of having additional resources, such as multiple processors. NOTE: Testers used this test environment to its maximum potential. The numbers they achieved with these tests provide a good inference pattern that can help companies decide on their ideal setups. To obtain accurate results and ensure their reliability, testers used the following methods:. Due to the controlled nature of the setup, testers considered alternative print methods that simulate a print job submission from the iprint mobile app to the iprint Appliance. The job submission mechanism must be tested in a controlled environment to calculate accurate reference values. Testers did not consider email jobs because several factors can lead to inconsistent email transaction times in enterprise email subsystems, thereby causing inaccuracy in reference values.. Testers recorded the time until print gateways submitted the jobs to the printers. Because print time depends on the printer s make and model, testers did not account for this time. 3. The test bed and setup used a GBps LAN network. 4. Testers used VMware ESXi 5.5 for deploying all appliances in the test. 5. Testers used the Windows 7 Enterprise 64-bit operating system created as a virtual machine for configuring external renderers. 6. Testers submitted jobs using iprintman via IPP to emulate normal-load and peak-load job traffic. 7. Testers used audit logs for each participating printer to capture these parameters: a. Job_submission_time b. Job_completion_time Using a and b, testers calculated: c. Wait time for jobs d. Total submission time 8. Peak load tests submitted jobs continuously until the jobs reached a count limit. 9. Testers also used mixed sets of jobs that included both types of loads (normal and peak) for file types belonging to both internal and external renderers. The ratio of each category was as follows: a. percent jobs for external renderer (5 percent PDF and 5 percent Microsoft Office documents) b. 4 percent jobs for internal renderer (75 percent images and 5 percent Open Office documents) NOTE: Testers did not perform management functions with Management Console or imanager when tests were running. 5. Test Lab Design Test Bed Internal Configuration Tools Client Servers Number of scenarios tested IMPORTANT: Make sure that iprint Appliance running in ESX server has sufficient physical memory available. As a best practice, the ESX server hosting this appliance should have around GB of additional free memory. Server Hardware Details Testers used machines with single-core and dual-core CPUs for testing: NOVELL iprint APPLIANCE Table 5-: Server Specifications Test Server Server Role Make and Model CPUs/Cores Memory Operating System Internal renderer Windows 7 Enterprise 64-bit configured with Microsoft Office Standard Edition Adobe Acrobat X Pro Symantec antivirus client iprint audit logs Modified iprintman for job submission in five different servers Six iprint Appliance Appliance Server Open Virtualization Format (OVF) deployed in VMware ESX running on Dell PowerEdge R5 4 GB Modified Novell Open Enterprise Server SP with mobile iprint RPM 4
Table 5-: ESX Server Hardware Details Table 6-: Results CPU Memory Hard Disk LAN Speed EXTERNAL RENDERER Table 5-3: Workstation Specifications Intel Xeon.7 GHz processor 6 processor cores per socket 6 GB 95 GB GBps Number of Jobs per Minutes Print Time Wait Time for th Job Print Time Wait Time for th Job 3 Print Time Wait Time for 3th Job 3 Test Server Server Role Make and Model Windows renderer Virtualized Windows 7 in VMware ESX abstracted on Dell PowerEdge R5 5 Print Time Wait Time for 5th Job 75 Print Time Wait Time for 75th Job 88 3 3 77 7 73 CPUs/Cores Memory 4 GB Print Time Wait Time for th Job 74 5 9 47 64 Operating System Windows 7 Enterprise 64-bit on ESX server Table 5-4: ESX Hardware Configuration CPU Memory LAN Speed Intel Xeon.4 GHz processor 4 processor cores per socket 8 GB GBps 6. Tested Scenarios Unless otherwise specified, the printing and wait times listed in the following tables are in minutes, rounded to the nearest whole minute. Printing Time for Set of Jobs for Multiple Windows, with Fixed Period of Minutes Testers performed this test with the following variables: Mixed set of different jobs: 5 PDFs ( MB, MB, 3 MB, 4 MB, 5 MB) DOC (5 KB, MB) XLS ( KB) PPT ( MB, MB) Number of external renderers: Tested with, and 3 Number of jobs: Tested with,, 3, 5, 75 and jobs submitted over an hour www.novell.com 5
File and Networking Services White Paper Performance Test Report: Novell iprint Appliance. Print Time for Mixed Set of Jobs (Internal and ), Normal Load for Different Job Volumes Testers performed the test with the following variables Mixed set of different jobs: 3 PDFs (3 MB, 4 MB, 5 MB) DOC ( MB) XLS ( KB) PPT ( MB) ODT ( KB) 3 images (JPG, TIF, PNG 3 MB, 5 MB, 5 MB) Number of external renderers: Tested with Number of jobs: Tested with, 5, 75 and jobs submitted over an hour Print Time for PDF Jobs, with Fixed Period of Minutes, for Different Job Volumes Testers performed this test with the following variables: Number of files: 8 PDFs Size of jobs: KB, 5 KB, MB, MB, 3 MB, 4 MB, 5 MB and MB Number of jobs: Tested with jobs submitted over an hour Table 6-3: Results Number of Jobs per Minutes Print Time Wait Time for 75th Job Internal 69 Built-in PDF 53 86 Adobe Acrobat X Pro 3 63 Table 6-: Results Number of Jobs per Minutes Print Time Wait Time for Last Job 5 8 75 3 6 6 98 Print Time for Mixed Set of Nine Jobs for Client and Server Versions of Windows, with Fixed Period of Minutes Testers performed this test with the following variables: Mixed set of nine different jobs: 3 PDFs (3 MB, 4 MB, 5 MB) DOC (MB) XLS ( KB) ODT ( KB) 3 images (JPG, TIF, PNG 3 MB, 5 MB, 5 MB) 6
Number of external renderers: Tested with, and 3 Number of jobs: Tested with, 5, 75 and jobs submitted over an hour Table 6-4: Results Number of Jobs per Minutes Print Time Wait Time for th Job 5 Print Time Wait Time for 5th Job 75 Print Time Wait Time for 75th Job Print Time Wait Time for 75th Job (Windows Server 8 R) Seconds 4 5 89 s4 (Windows 7) 3 Seconds 9 35 35 7 88 5 Average Print Time for Different File Formats and Different File and Page Sizes The following table provides an idea of the time it took iprint Appliance to print different file formats: Table 6-5: File Formats Total Print Time File Size File Formats Total Pages in Seconds KB ODT 5 8 KB XLS 63 MB DOC 5 87 3 MB PDF 94 4 MB PDF 6 3 5 MB PDF 8 6 3 MB TIF 9 5 MB JPEG 98 5 MB PNG 7 7. Comparisons Testers have compared iprint Appliance. and iprint Appliance.. They compared the following two attributes from the tests, keeping all other attributes (job load and document format) constant. Total print time Total wait time for last job With most organizations, company policies mandate security applications like antivirus, monitoring tools and so forth. Testers found that antivirus client scans are the most CPU and file I/O intensive, so testers checked the impact on rendering performance while an antivirus scan was running. They concluded that with a single core system, data indicates performance degradation, whereas with a dual-core system, the results were comparable to the results seen for tests run without antivirus scans running. The internal and external renderers in iprint Appliance. have been improved since iprint Appliance.. In iprint Appliance., the internal renderer has been updated with improved software and the external renderer now comes with a built-in PDF renderer. This is an improvement from iprint Appliance., in which Adobe Acrobat Pro was required by the external renderer to render PDF documents. Because of these improvements, testers only compared results for printing with Windows renderers, as the changes between the two versions mean further testing to compare iprint Appliance. and. would not be valid. Unless otherwise specified, the print and wait times listed in the following tables are in minutes, rounded to the nearest whole minute. www.novell.com 7
File and Networking Services White Paper Performance Test Report: Novell iprint Appliance. Comparison of iprint Appliance. and iprint Appliance., Print Time for Set of Jobs for Multiple Windows, with Fixed Period of Minutes Testers performed this test with the following variables: Mixed set of different jobs: 5 PDFs ( MB, MB, 3 MB, 4 MB, 5 MB) DOC (5 KB, MB) XLS ( KB) PPT ( MB, MB) Number of external renderers: Tested with, and 3 Number of jobs: Tested with,, 3, 5, 75 and submitted over an hour Table 7-: Results Number of Jobs per Minutes.... 4. 3 7 5 6 88 63 7 75 7 3 94 73 34 74 95 9 69 64 3. Comparison of iprint Appliance. and iprint Appliance., Wait Time for Last Job, Set of Jobs for Multiple Windows, with Fixed Period of Minutes Testers performed this test with the following variables: Mixed set of different jobs: 5 PDFs ( MB, MB, 3 MB, 4 MB, 5 MB) DOC (5 KB, MB) XLS ( KB) PPT ( MB, MB) Number of external renderers: Tested with, and 3 Number of jobs: Tested with,, 3, 5, 75 and submitted over an hour Table 7-: Results Number of Jobs per Minutes.... 3. 3 5 56 3 3 75 77 34 74 5 65 4 9 4. 8
8. Conclusions Novell classifies iprint Appliance performance into three different categories. Each category has an average-load and peak-load handling capability which have been derived from the test results described above for maintaining Quality of Service (QoS). QoS refers to the wait time to print a job. Testers calculated average loads for job submissions in a -hour window per day. The loads calculated for increased job-submission rates over short spans of time are termed peak loads. Section 4 Test Methodology and Section 5 Test Lab Design mention the test environment and methodologies. In the following table, the time iprint Appliance took to render a single print job ranges from -8 seconds, depending on the average load for each category. For example, if your print environment falls into Category as defined in the table, and the average load is up to 5 print jobs per day, then the time taken to render a single print job is one second. As the average load increases to jobs per day, the time taken to render a single print job increases, and can go up to 8 seconds depending on the job size. Testers saw improved performance for iprint Appliance. compared with iprint Appliance. in all three of the categories. Novell iprint Appliance modularity has the ability to maintain rendering time under acceptable limits by scaling up external renderers. Testers achieved better rendering performance when they optimized wait times for increased job volumes by adding external renderers. Average-load test results with single and multiple renderers define iprint Appliance job-handling capabilities, which can help administrators determine the number of external renderers your organization will require for optimally handling print loads within acceptable job wait times. Peak-load test results provide an overview of the wait time for the last job for single and multiple renderers with varying job volumes. These results can help administrators more precisely determine the number of external renderers your organization will need, depending on the occurrence of peak loads in your environment. Based on the data they collected for the most likely printing scenarios, Novell testers have plotted charts and graphs to help your system administrators understand external renderer requirements for an optimal rendering experience. For more information, see the Novell iprint Appliance Administration Guide at: www.novell.com/documentation/iprint-appliance/ Table 8-: Load Handling Capability, iprint Appliance. Category Setup Average Load Peak Load Category iprint Appliance + Category iprint Appliance + Category 3 iprint Appliance + 4 7 jobs per day 4 5 jobs per day 5, jobs per day 7 jobs in minutes 4 5 jobs in minutes 5 jobs in 3 minutes www.novell.com 9
By engaging Novell Services for Premium Support, Consulting or Training, we can help you get the most of your product investment to suit your business needs. Please contact us today, or contact your local Novell Solutions Provider: Premium Support and Consulting: 8 74 34 U.S./Canada 8 86 47 Worldwide crc@novell.com Training: 8 33 338 U.S./Canada 8 86 338 Worldwide training@novell.com Novell, Inc. 8 South Novell Place Provo, UT 846 USA www.novell.com 46-- /5 5 Novell, Inc. All rights reserved. Novell and the Novell logo are registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.