Altiris Software Package Multicast Performance Testing

Transcription

1 Altiris Software Package Multicast Performance Testing Prepared By: Heath Ramsey, Sr. Consultant June 27, Software Package Multicast Performance Testing> 1

2 About Altiris Altiris, Inc. is a pioneer of IT lifecycle management software that allows organizations to easily manage desktops, notebooks, thin clients, handhelds, industry-standard servers, and heterogeneous software including Windows, Linux and UNIX. Altiris automates and simplifies IT projects throughout the life of an asset to reduce the cost and complexity of management. Altiris client and mobile, server, and asset management solutions natively integrate through a common Web-based console and repository. For more information, visit NOTICE The content in this document represents the current view of Altiris as of the date of publication. Because Altiris responds continually to changing market conditions, this document should not be interpreted as a commitment on the part of Altiris. Altiris cannot guarantee the accuracy of any information presented after the date of publication. Copyright 2005, Altiris, Inc. All rights reserved. Altiris, Inc. 588 West 400 South Lindon, UT Phone: (801) Fax: (801) BootWorks U.S. Patent No. 5,764,593. Altiris and Deployment Solution for Servers are registered trademarks of Altiris, Inc. in the United States. Microsoft, Windows, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries. Other brands and names are the property of their respective owners. Information in this document is subject to change without notice. For the latest documentation, visit TWPT: B 2 < Software Package Multicast Performance Testing

3 Contents Introduction...5 What This Paper Will Cover... 6 What This Paper Will NOT Cover... 6 The Test Lab Environment...8 Network Topology... 8 Microsoft Active Directory Configuration... 9 Client Workstation Hardware...10 Server Hardware...10 Altiris Infrastructure Configuration...11 Testing Methodology...15 Test 1 Notification Server Communications Baseline...17 Expected Outcome...18 Test Results...18 Conclusion and Trend Information...18 Test 2 Package Server Communications Baseline...19 Expected Outcome...19 Test Results...19 Conclusion and Trend Information...20 Test 3 Multicast Communications Baseline...21 Expected Outcome...21 Test Results...21 Conclusion and Trend Information...22 Test 4 Multicast Communications with Varying Package Sizes Expected Outcome...23 Test Results...24 Conclusion and Trend Information...28 Test 5 Multicast Communications with Varying Package Sizes Expected Outcome...29 Test Results...30 Conclusion and Trend Information...34 Test 6 Multicast Communications with Varying Numbers of Agents...35 Expected Outcome...36 Test Results...36 Conclusion and Trend Information...40 Test 6A Multicast Communications with Varying Client Config Interval...41 Expected Outcome...42 Test Results...42 Conclusion and Trend Information...43 Test 7 Multicast Communications Across Multiple Microsoft Domains...43 Expected Outcome...44 Test Results...44 Conclusion and Trend Information...44 Test 8 Multicast Communications with Varying Concurrent Sessions Software Package Multicast Performance Testing> 3

4 Expected Outcome...45 Test Results...45 Conclusion and Trend Information...47 Test 8A Multicast Communications with Varying Maximum Sessions per Physical Subnet Settings...48 Expected Outcome...48 Test Results...48 Conclusion and Trend Information...50 Test 9 Multicast Communications with Multiple Package Delivery...50 Expected Outcome...51 Test Results...51 Conclusion and Trend Information...52 Conclusion...53 Revision History < Software Package Multicast Performance Testing

5 Introduction One of the main goals for using Altiris software to manage software delivery in the enterprise is the efficient delivery of software packages to workstations that need them. Altiris Client Management Suite (CMS) has allowed customers to easily and effectively distribute software around the enterprise. However, the implementation of CMS in a complex environment requires a level of custom configuration and design to ensure the impact of software delivery traffic on the network does not adversely affect the performance of that same network for other applications and data needed the communications bandwidth. Just installing CMS with the default configuration can lead to some undesired side-effects in complex network infrastructures. One of the main mechanisms for shaping software delivery traffic through CMS is the implementation of Package Servers on the network. By using Package Servers, an administrator can explicitly exert influence over how individual workstations will download software packages on the network. The disadvantage of using Package Servers is the administrative overhead required to implementation, update, and maintain the Package Server infrastructure. For most organizations, Package Server implementation would not require a significant amount of overhead. However, there are some organizations that would need to implement hundreds of Package Servers to control network traffic. For those organizations, the administrative overhead can be as costly as the original traffic problem. With the release of Service Pack 2 for Notification Server, Altiris has introduced a new mechanism administrators can use for controlling software package traffic on the network. This new communications mechanism uses TCP/IP multicasting to deliver software packages across the enterprise. While multicast traffic with Notification Server is new, Altiris is familiar with multicast as a technology. Altiris has been using multicast for years as a means to make network communications more efficient. With Deployment Server (a component of Deployment Solution), multicast can be an integral part of image delivery in the enterprise. TCP/IP multicast is a means for individual hosts to become part of a larger group for packet transmission purposes. Instead of attempting to broadcast a set of packets, the packet is sent to a special range of addresses on which all members of the group are listening. By using this form of group communication, a packet can be transmitted once by the sender yet concurrently received by all intended recipients. When multicast is implemented properly, it is more efficient in the delivery of packets to recipient machines than the conventional unicast (individual host directed packets) or broadcast (delivered to all hosts) mechanisms. So while Altiris is not new to multicast, multicast is new to Notification Server and software delivery. It is important to understand how the Altiris Agent is using multicast. It is also important to understand some of the key multicast configuration settings for the Altiris Agent and their impact on the network from a communications perspective. The purpose of this paper is to describe the results of some basic testing that was performed to show how multicast can impact a customer environment. Software Package Multicast Performance Testing> 5

6 What This Paper Will Cover The topic of multicast with respect to the Altiris Notification Server is a broad one. To that end, the paper will focus on some very specific topics dealing with the testing. The points to be covered by this paper are summarized in the list below: This paper will discuss the results of the various tests conducted during the one week of lab time allotted to the testing process. These results will detail the test conducted, the intended purpose of the test, the generally observed traffic patterns, and any pertinent statistics and measurements that could be gleaned from the test results. This paper will provide data points for a specific controlled environment. From these known data points, a person can begin to extrapolate expected behavior patterns. It is by no means meant to be a definitive resource for design guidelines. It will be a single information point to be used in conjunction with other known data points to realize a larger design for specific customer implementations. This paper will make some broad conclusions based on the testing results that will need further field research and testing to confirm or refute. For right or wrong, these conclusions will be accurate based on the limited information collected during the testing process. At the end of each test, a statement of trend information will be made. It is these statements that can be used as data points in customer specific design. What This Paper Will NOT Cover Because there is limited information regarding the multicast implementation in the Altiris Notification Server, there is much the general reader might wish to know. This document is not intended to be a comprehensive discussion of the Altiris Notification Server multicast implementation. This document will NOT be covering the following topics: This document will not provide a general discussion of TCP/IP multicast. It is assumed the reader has a more-than-working knowledge of both the TCP/IP protocol and its multicast properties. For a more technical discussion of multicast please refer to <insert multicast reference here>. This document will not provide the reader a step-by-step approach to configuring multicast in the Altiris Notification Server. This information is already provided through the Notification Server documentation. For information on configuring multicast for Notification Server, please refer to the SP3 Release Notes found at the Altiris Support Documentation website. This document is not meant to replace the developer/engineering recommendations made in Brent Bishop s multicast white paper titled Multicast Concerns with Default Settings. It is a known fact that the default multicast configuration in the product is not appropriate for customer environments. Any multicast implementation will require a modification to the default settings to tune the product to the environment. 6 < Software Package Multicast Performance Testing

7 The results that will be detailed in this document are based upon one (1) week of testing in a controlled lab environment. Each test described in this document was performed once, and the raw data was collected after test completion. The raw data was analyzed after leaving the testing site; therefore, no re-testing was performed to correct any testing methodology failures. Any points of data appearing to be statistical outliers are called out in this document. The testing results contained in this document should be verified and corroborated through future testing sessions. However, the data itself should be used as a starting point for examining the impact of the multicast configuration on the network. This data can then be used as one of the tools in the consultant s toolkit as he or she develops a software package delivery infrastructure design for a particular customer site. Software Package Multicast Performance Testing> 7

8 The Test Lab Environment As stated, the lab environment used for testing was a simple network. Two network segments were separated by a router. This router was used to isolate multicast traffic in the lab environment. The Altiris servers supporting the infrastructure were located on one side of the router. The workstations with the Altiris Agent were located on the opposite side of the router. Each component of the test lab will be examined more thoroughly to help understand the impact of the hardware and network used on the test results. Network Topology The network topology for the test consisted of 2 TCP/IP subnets connected to each other by a router. The networks on each side of the router were 100MBps switched networks. Figure 1 illustrates the network topology of the lab environment network mask network mask ` Notification Server Client Workstations ` ` Database Server Figure 1: Test lab network topology. Each network was assigned a 10.x.0.0 address in a TCP/IP class B addressing scheme. All client workstations existed in the same broadcast domain from a TCP/IP perspective. The router between the two networks was not configured to pass any multicast membership traffic. Therefore, the client workstations also existed in the same multicast domain. However, the Notification Server itself was not part of that multicast domain. This was intentionally done to ensure any multicast master from the Altiris perspective would be a workstation local to the network. From a TCP/IP configuration perspective, DHCP was enabled on the client network ( ), and this is how the machines on their network received TCP/IP addresses. The only exceptions to 8 < Software Package Multicast Performance Testing

9 this configuration on the network were the two (2) machines configured as Active Directory Domain Controllers. In order to facilitate name resolution, DNS servers were set up in conjunction with Active Directory. The Microsoft Active Directory Domain configuration will be detailed in the next section of this document titled Microsoft Active Directory Configuration. The network also required Windows Internet Naming Service (WINS) to be configured for name resolution. This was to help facilitate resource discovery and agent rollout across the network. The router prevented the browsing of NetBIOS names by the Notification Server. Therefore, the database server was configured to be the primary WINS server for the entire network. This is how the domain membership lists and domain browse lists were used to discover resources and push out the Altiris Agent to the machines in the test network. Microsoft Active Directory Configuration Part of the testing that was completed dealt with software package delivery to machines in separate Microsoft security domains. To facilitate this test, the machines were divided up into three (3) domains. From an Active Directory perspective, three (3) separate trees were created. This would ensure that no security credentials were shared between machines and skew any test results pertaining to security. Three (3) machines acted as Active Directory Domain Controllers to support the separate domains. Each Domain Controller was set up in Active Directory mixed mode. Table 1 documents the domain names and the number of machines installed in each security domain. Microsoft Domain Name Domain Controller Number of Member Workstations SERVERS BL20PG3-B 68 CLIENT1 DSR03C11 33 CLIENTS2 DSR03C10 17 Table 1: Microsoft Domain Membership. Each Domain Controller also had DNS installed as part of the Active Directory set up. Client workstations installed into a domain used the Domain Controller as its primary DNS server. Figure 2 shows the relationship between the Active Directory Domain Controllers, DNS, and the workstations installed into those domains. Software Package Multicast Performance Testing> 9

10 Figure 2: Domain members retrieve DNS information from the Domain Controller The lab itself did not have any Internet connectivity. Therefore, any DNS requests for machines names stopped at the Domain Controllers because the root DNS servers were not available to the Primary DNS servers for the zone. From a DNS perspective, workstations in a specific Microsoft security domain could resolve only other machine in their domain. Members of other domains could not be resolved through DNS because the DNS servers for each domain did not know about each other. The configuration of the Domain Controller machines will be discussed later in this document in the sections Client Workstation Hardware and Server Hardware. Please refer to those sections for information about hardware and operating system specifications for the Domain Controllers. Client Workstation Hardware In total, 116 workstations functioned as an Altiris client and were connected to the network for testing purposes. Of these 116 machines, two (2) machines were set up as Domain Controllers for the infrastructure. While the Domain Controllers had a Windows Server OS, they were treated as normal Altiris Agents for the purposes of the testing. Two different manufacturer machines were used in the lab for testing. Both Dell and HP machines were utilized in the test bed. The hardware configuration for the client workstations was not consistent for a single manufacturer. This was done to represent the various types of machines that would be present in a normal customer environment. All machines that were not functioning as Domain Controllers ran the Windows XP operating system with Service Pack 1 applied. The Domain Controllers were running Windows Server operating systems. Server Hardware Two (2) servers were used to support the Altiris Infrastructure in the test lab. One server functioned as the communications infrastructure and database server. The second server was used 10 < Software Package Multicast Performance Testing

11 solely for the Altiris Notification Server. The database and NS were installed on different machines because this is a more common configuration that is being found in customer sites. Server class hardware comparable to what would be found at a customer site was used for the test lab. Each server was a dual processor box with 2GB RAM. This configuration was more than sufficient to handle the 100+ machines in the lab environment from a software package delivery perspective. Table 2 details the hardware configuration and OS/communication services installed on each machine. Notification Server Database Server Manufacturer: HP Manufacturer: HP Name: BL20PG3-A Name: BL20PG3-B OS: Windows 2003 Server Enterprise Edition OS: Windows 2003 Server Enterprise Edition Processor: 2 Xeon 3.4GHz Processor: 2 Xeon 3.4GHz Memory: 2GB RAM Memory: 2GB RAM Disk Space: 16.94GB Disk Space: 16.94GB Network Connectivity: 100MBps Network Connectivity: 100MBps IP Address: IP Address: Domain: SERVERS member machine Domain: SERVERS DC (Mixed-mode) Services: Notification Server Services: SQL Server 2000 SP3 WINS DNS Table 2: 2 : Test Lab Server Hardware Configuration The servers themselves did have multiple network cards installed; however, the extraneous interfaces were not used. The Notification Server, BL20PG3-A, did have temporary Internet connectivity to download the Altiris software for the installation of the Altiris infrastructure; however, this connectivity was removed at the commencement of testing. Altiris Infrastructure Configuration As stated, the services required for supporting the Altiris infrastructure were split across two (2) servers. The first server, BL20PG3-A, had Notification Server with Service Pack 2 installed. The second server, BL20PG3-B, served as the database server for the infrastructure and had SQL Server 2000 SP3 installed. Because the testing was focused on the multicast delivery of software packages to managed machines, a limited set of solutions were installed on the Notification Server. Subsequently, a small set of solution agents were rolled out to the managed machines on the network. Table 3 documents the solutions installed on the Notification Server and the solution agents installed on the managed workstations. Software Package Multicast Performance Testing> 11

12 Notification tion Server Installed Components Notification Server Version: Inventory Solution Version: 6.0 Software Delivery Solution Version: 6.1 Patch Management Solution Version: 6.0 Rolled-out out Altiris Agent Components Altiris Agent Version: Altiris express AClient Version: Software Delivery Solution Agent Version: Task Synchronization Agent Version: Table 3: 3 : Altiris Solutions Installed in Test Lab Environment There are a few items in Table 3 that bear further explanation, as the solutions installed on the Notification Server do not completely match the agent components installed on the managed workstations. The discrepancies between the two lists are explained in the summary below: Inventory Solution 6.0 was installed on the Notification Server; however, no policies were enabled for inventory. Inventory was installed with the thought of harvesting all configuration information for the purposes of documentation in this paper. Unfortunately, there was not enough time to enable the Inventory policies after the multicast testing had completed. Patch Management Solution 6.0 was installed on the Notification Server but went unused. The original intent was to use Microsoft patches as the packages for multicast delivery; however, other packages were used for delivery instead of the patches. The managed workstations had the Altiris express AClient installed even though Deployment Solution was not installed on the Notification Server. There was a separate Deployment Server in the lab environment that was used to set up and configure the managed workstations in the lab. AClient was not part of the testing process but had to be present on the machines. NOTE: Since the original multicast testing was conducted there have been en a number of upgrades to the various Altiris infrastructure solution components; however, the core multicast engine in the Altiris Agent has not been modified. The test results documented in this paper are completely valid with the most current release of the Altiris Agent with Notification Server SP3. The Altiris Agent itself was pushed out to the client workstations using the Altiris Console. Once installed, the agent used mostly the default configuration. However, there were some settings that were changed to facilitate the testing process. Tables 4-8 document the base Altiris Agent configuration settings used in the test lab. General Tab Setting Applies to Collections Request New Configuration Information Every Send Basic Inventory Every Agents Should Compress Events Over Run from the Server if Bandwidth is Above Download and Run Locally if Bandwidth is Above Download the Package Files as Soon as Possible Value All Desktop Computers (excluding Package Servers ) 15 minutes (testing only) 1 day (Recommended) Enabled/200Kbytes (Recommended) Disabled Enabled/Any Connection Speed Enabled 12 < Software Package Multicast Performance Testing

13 General Tab Setting Download the Package Files Before Running the Program Default Minimum Connection Speed to Run SWD Packages Never Throttle Enable Throttling When Connection Speed is Below Throttle Regardless of Connection Speed Use the Server s Time for Throttling Settings Add Throttling Settings Add Blockout Settings Value Disabled No network connection required Disabled Enabled/50 KByte/sec Disabled Disabled No Throttling Defined No Blockout Defined Use the Server s Time for Blockout Settings Disabled Table 4: 4 : Base Settings for the General Tab in the Altiris Agent Configuration Policy Interaction Tab Setting Display a System Tray Icon on the Altiris Agent Computers Notify Users When Scheduled Software Delivery Tasks Arrive (for 5.x Agents Only) Notify Users When Manual Software Delivery Tasks Arrive (for 5.x Agents Only) Show Software Delivery Options on the Altiris Agent Value Enabled Disabled Enabled Enabled Notify in Advance Do not notify Table 5: 5 : Base Settings for the Interaction Tab in the Altiris Agent Configuration Policy Package Multicast Tab Setting Allow Altiris Agents to Use Multicast for Downloading Packages By Default the Altiris Agent Should Use Multicast When Downloading Packages Value Enabled Enabled Maximum Master Sessions Per Computer 2 Minimum Receiving Computers Per Session 1 Wait Time to Begin Session Number of Receiving Computers Required to Begin Session Before Wait Time Has Expired Maximum Bandwidth to Use for Multicasting 50% of Configuration Request MBytes/sec Maximum Receiving Attempts Per Package 3 Maximum Sessions Per Physical Subnet 10 Disable Multicast for Packages Smaller Than 512 Kbytes Table 6: 6 : Base Settings for the Package Multicast Tab in the Altiris Agent Configuration Policy Software Package Multicast Performance Testing> 13

14 Advanced Settings Tab Setting Specify an Alternate URL for the Altiris Agent to Use to Access the NS Value Disabled Enable Power Management on Altiris Agents Disabled Table 7: 7 : Base Settings for the Advanced Settings Tab in the Altiris Agent Configuration Policy General Tab Setting Value TCP/IP Port TCP/IP Multicast Address TCP/IP Multicast Port TCP/IP Multicast Address TCP/IP Multicast Port TCP/IP Listener Range TCP/IP Exclusion Range No TCP/IP Exclusion range defined Table 8: Base Settings for the General Tab in the Global Altiris Agent Settings Policy Any deviations in the Altiris Agent configuration will be documented in the section describing the test and the test results. If no specific mention is made to the modification of a particular setting, it is assumed the value of that setting listed in the base settings of Tables 4-8 applies to that test. 14 < Software Package Multicast Performance Testing

15 Testing Methodology The focus of all testing performed in the lab environment was the delivery of software packages via multicast to managed workstations. Each test attempted to vary only a single variable in the multicast configuration while keeping all other variables constant. By doing so, trending tendencies can be discovered as the value on the axis of the variable changes. There are advantages and disadvantages to this testing approach. The main advantage of this approach is the data quality regarding the tested variable. A result from the change in the variable can be attributed specifically to that change. In other words, there is a distinct cause and effect result that can be measured through the isolation. By documenting the changes and the trends, the data can be extrapolated to larger values. The main disadvantages to this approach deal with large scale extrapolation and variable synergy. From an extrapolation perspective, the extrapolation will generally hold true based on the steady state of the data. However, there are no guarantees the trend is true beyond the scope of the data set. Any scenario that falls outside of the data set is truly an unknown case from the testing perspective. The observed results may or may not be in line with the extrapolated data. For example, a trend might be observed as the number of managed workstations participating in the multicast session increases. The trend and performance can be documented for the 116 machines in the lab environment. This trend may hold true for 200, 500, and 1000 machines. However, it may break down at 1500 or 2000 machines. This breaking point can t be known unless empirically testing with an appropriate test bed. Therefore caution needs to be used when extrapolating the data outside of the data set. The second disadvantage to the testing approach is variable synergy. The data points are being plotted on the individual axis of the modified variable. The variables themselves are being examined in turn. However, the picture being created is a focused and narrow two dimensional picture. It does not predict the result when two variables are modified in concert. There were times during the test runs when multiple variables were modified in an attempt to perceive if there was any variable synergy occurring. Those instances will be documented as part of the testing results in the specific test section. There were six (6) scenarios examined through the testing process. These tests were conducted after a set of baseline tests were run. The purpose of the baselines was to model and measure normal software package delivery between the Notification Server and managed workstations and between Package Servers and managed workstations. From there, the multicast scenarios could be staged and the data gathered. Software Package Multicast Performance Testing> 15

16 In all, the testing scenarios are summarized in the following list: 1. Baseline software package delivery from the Notification Server 2. Baseline software package delivery from a Package Server 3. Baseline software package delivery through multicast 4. Multicast software package delivery and varying package sizes where package is not present 5. Multicast software package delivery and varying package sizes where package is present 6. Multicast software package delivery and varying number of managed workstations 7. Multicast software package delivery across multiple Microsoft security domains 8. Multicast software package delivery and varying number of concurrent multicast sessions on the wire 9. Multicast software package delivery with multiple package delivery required In an effort to measure performance and observe the network traffic model, packet sniffers were used. The tool of choice for the test lab is Ethereal version Ethereal was installed on the Notification Server and select managed workstations in the network. Because the network itself is a switched network every packet could not be captured in this approach. However, if the sniffing machine participated in the multicast session all of the multicast traffic would be captured because of the nature of multicast traffic. Figure 3 shows the placement of the Ethereal packet sniffer software on the test network. In performing the tests, great care was taken to ensure the various multicast sessions would be captured as they were happening on the wire. 16 < Software Package Multicast Performance Testing

17 Figure 3: Test Lab Ethereal Placement Using the information captured in the packet traces, the following information can be obtained for the results of each test: Time elapsed for initial package delivery (i.e., getting the package to the first machine requesting it) Time elapsed in delivering a package to all machines participating in the test Time elapsed for the multicast negotiation process to complete for a package request Time elapsed for the multicast delivery for a package Time elapsed in downloading the requested package from the Notification Server (if required) General traffic model and network activity in delivering the requested package For each test, there will be an explanation of the test, a summary of the results based on the above metrics, and a section explaining the test results and providing trend information based on those results. Test 1 Notification Server Communications Baseline The purpose of this test is to provide some basic metrics for a single managed workstation downloading packages of various sizes from the Notification Server. In this test, five (5) different packages were created on the Notification Server. The requisite Software Delivery Tasks were created to deliver the package to a managed workstation. Each task was enabled in turn such that only one task was active on the wire. Only after one task had completed would the next task be enabled. Software Package Multicast Performance Testing> 17

18 The packages being delivered to the workstation were very simple. The packages themselves consisted of a single file. The command line for the packages was the opening of an Explorer browser on the managed machine. This provided for the creation of packages that could be easily re-usable and distributed. For the purposes of this test, the sizes of the packages were roughly 10K, 100K, 1MB, 10MB, and 50MB. For this test multicast package delivery was disabled. This would revert the infrastructure to what is considered to be normal communications between the Notification Server and the Altiris Agent. Expected Outcome The expected outcome for this test is to observe unicast TCP/IP traffic between the managed workstation and the Notification Server. The variable being modified for this test is the size of the package being delivered. Given that the package is increasing in size, it is expected to take longer for larger packages to be delivered to the workstation. An increase in the size of the package leads to an increase in the time of delivery. Test Results Table 9 summarizes the results of the package download performance for the baseline test. Package Size Time to Obtain Package Information Time to Download 10K 1.45s 0.11s 100K 1.68s 0.34s 1M 1.87s 0.89s 10M 2.10s 11.69s 50M 2.23s 41.37s Table 9: 9 : Notification Server Package Download Baseline Data In Table 9, the Package Size column represents the size of the software package downloaded by the managed workstation. The Time to Obtain Package Information column is the amount of time elapsed between requesting the client configuration policy and actually starting the software package download. The Time to Download column is the amount of time spend on the wire downloading the package to the managed workstation. As expected, the communications traffic seen in the packet sniffer is unicast between the managed workstation and the Notification Server. There were no unexpected behaviors or especially interesting information that can be gleaned from the packet trace. Conclusion and Trend Information As expected, there is basically a linear relationship between the size of the package and the amount of time it takes to download the package from the Notification Server. Larger software packages will take longer to download from the Notification Server than smaller software packages. 18 < Software Package Multicast Performance Testing

19 One unexpected result of the test appears to be a relationship between the size of the package and the amount of time required to obtain the package information. Each package in this test consists of a single file in the software package of comparable name length. While the overhead is relatively negligible, it is an interesting relationship that may require further study and analysis. This test is obviously quite simple and trivial. There is no extraneous load on the Notification Server, and there is but a single managed workstation downloading the software package. However, it does show in an effectively scaled environment on a simple network that software package download from the Notification Server is probably the most efficient mechanism for distribution. Test 2 Package Server Communications Baseline The purpose of this test is similar to the previous test. The test provides some basic metrics for a single managed workstation downloading packages of various sizes from a Package Server. In this test, five (5) different packages were created on the Notification Server and staged to a Package Server located on the same network as the managed workstation. The requisite Software Delivery Tasks were created to deliver the package to a managed workstation. Each task was enabled in turn such that only one task was active on the wire. Only after one task had completed would the next task be enabled. The packages being delivered to the workstation were very simple. The packages themselves consisted of a single file. The command line for the packages was the opening of an Explorer browser on the managed machine. This provided for the creation of packages that could be easily re-usable and distributed. For the purposes of this test, the sizes of the packages were roughly 10K, 100K, 1MB, 10MB, and 50MB. For this test multicast package delivery was disabled. This would revert the infrastructure to what is considered to be normal communications between the Notification Server, the Package Server, and the Altiris Agent. Expected Outcome The expected outcome for this test is to observe unicast TCP/IP traffic between the managed workstation and the Notification Server so the workstation can obtain the information about the package and the Package Server hosting the package. Once that information is obtained, the managed workstation should use unicast TCP/IP traffic to contact the Package Server and download the package. The variable being modified in this test is the size of the package being delivered. Given that the package is increasing in size, it is expected to take longer for larger packages to be delivered to the workstation. An increase in the size of the package leads to an increase in the time of delivery. Test Results Table 10 summarizes the results of the package download performance for the baseline test. Software Package Multicast Performance Testing> 19

20 Package Size Time to Obtain Package Information Time to Download 10K 5.96s 3.08s 100K 1.65s 1.01s 1M 1.70s 1.59s 10M 1.87s 13.02s 50M 2.07s 47.40s Table 10: : Package Server Package Download Baseline Data In Table 10, the Package Size column represents the size of the software package downloaded by the managed workstation. The Time to Obtain Package Information column is the amount of time elapsed between requesting the client configuration policy from the Notification Server and actually starting the software package download from the Package Server. The Time to Download column is the amount of time spend on the wire downloading the package to the managed workstation from the Package Server. As expected, the communications traffic seen in the packet sniffer is unicast between the managed workstation and the Notification Server. The traffic is also unicast between the managed workstation and the Package Server. One interesting behavior viewed in the packet trace is the Package Server speed test conducted by the Altiris Agent the first time it learned about the Package Server. When the Package Server was discovered for the first time, the Altiris Agent conducted a ping test to gauge the speed of the Package Server. This negotiation occurred during the process of obtaining the package information. This had a large impact on the performance of the 10K package download, as this was the first package requested from the Package Server. Also, there was some overhead required in setting up the communication path between the managed workstation and the Package Server over the Microsoft networking protocol, SMB. This overhead would be seen on the wire whenever a new session needed to be created and no cached connection could be reused by the workstation. Again, this had a large impact on the performance of the 10K package download, as it was the first package requested from the Package Server. Conclusion and Trend Information As expected, there is basically a linear relationship between the size of the package and the amount of time it takes to download the package from the Package Server. Larger software packages will take longer to download from the Package Server than smaller software packages. The data for the 10K package is skewed by the overhead required to set up communications with the Package Server. Because the 10K package was the first package downloaded in a sequence, the Altiris Agent was forced to execute a performance test and set up the initial SMB connection with the Package Server. This affected how quickly the package could be downloaded from the Package Server as compared to the other tests. 20 < Software Package Multicast Performance Testing

21 This test is again obviously quite simple and trivial. There is no extraneous load on the Notification Server or the Package Server, and there is but a single managed workstation downloading the software package. However, it does show in an effectively scaled environment on a simple network that software package download from a Package Server is an effective and efficient way to deliver packages on the network. Test 3 Multicast Communications Baseline The purpose of this test is to provide some basic metrics for a single managed workstation downloading a single package from a Package Server with multicast enabled. In this test, a single package of 1M was created on the Notification Server and staged to a Package Server on the same network as the managed workstation. The requisite Software Delivery Task was created to deliver the package to a managed workstation. The package delivered to the workstation was very simple. The package itself consisted of a single file. The command line for the package was the opening of an Explorer browser on the managed machine. This provided for the creation of a package that could be easily re-usable and distributed. For the purposes of this test, the size of the package was roughly 1MB. No settings on the Altiris Agent were modified from the baseline to complete this set of tests. Expected Outcome This test is probably the worst-case scenario for multicast. Multicast is at its most efficient when a single piece of information is being concurrently distributed to many hosts. In this case, only one managed workstation actually needs the package, but it is going to have to go through the motions of setting up a multicast session before actually downloading the package from the Package Server via multicast. Therefore, it should take at least 7½ minutes to download the package, as that number represents 50% of the Altiris Agent configuration request interval. On the wire, unicast sessions are expected to be created by the managed workstation to retrieve package information from the Notification Server and the Package Server. This should be followed be the negotiation for a multicast session. Both the Package Server and the managed workstation should participate in the multicast session since the Package Server already has a copy of the package. There are no variables being modified in this test because it is testing the basic multicast functionality. The amount of time for the package download should actually be a small fraction of the time required to wait for and set up the multicast session. Test Results Table 11 documents the results of a single package download to a single managed machine with multicast enabled. Software Package Multicast Performance Testing> 21

22 Communication Phase Elapsed Time Percentage of Total Elapsed Time Obtain package information 6.71s 0.87% Negotiate multicast session ( ) Negotiate multicast session ( ) s 30.51% s 67.63% Download package 7.58s 0.98% Totals s (12m 50.76s) 99.99% (.01 lost to rounding) Table 11: : Package Download via Multicast Baseline Data The results from this test confirmed the basic model of the traffic, but the amount of time taken to download the package from the Package Server via multicast is far greater than anticipated. From a traffic perspective, the managed workstation created unicast sessions with the Notification Server and the Package Server to obtain package information. Once that process was complete, a multicast negotiation session started on the main multicast address ( ). The only network traffic observed between the managed workstation and the Notification Server was the administrative traffic required to obtain information about the package and to post information about the status of the Software Delivery Task once it had completed. No package distribution traffic crossed the router between the two networks, which is a vitally important piece of information for environments with costly and/or slow WAN links. The unexpected result of this test is the large amount of time taken to actually begin the distribution of the package from the Package Server once the transition to the transmission address ( ) occurred. There appears to be a second negotiation session between the machines participating in the multicast session before the package download actually commences. Once the download starts, though, the time to download is relatively quick. Conclusion and Trend Information When attempting to download a package to a single managed workstation, it is far less efficient from a time and network traffic perspective to use multicast than a direct download from a Package Server or the Notification Server. The managed workstation will spend very large percentage of the time needed to download the package (90%+) waiting for the multicast session to start due to session negotiation. The actual percentage will vary depending upon the size of the package. As compared to the download times for the same size package in the previous two tests, it took only a small fraction of the time to download through a direct unicast session from the Notification Server or Package Server than it did to download via multicast (2.76s NS, 3.29s PS, s multicast). While this test represents a trivial situation, the results cannot be ignored if rapid delivery of software packages is required in an environment. In the overall scheme of things, the amount of time required to download a package via multicast is not unreasonable. However, from a scalability perspective it could be problematic when 22 < Software Package Multicast Performance Testing

23 attempting to have a managed workstation download many packages in a short period of time. The Altiris Agent only downloads packages sequentially. If there were 25 1M packages queued up to be downloaded by a managed workstation via multicast, it would take at least 5 hours to download all of them. With direct communications, the same packages could be downloaded in less than 100 seconds. These factors need to be taken into consideration when designing an infrastructure for a customer. From the results of this test, multicast should be used when there is a compelling reason to do so. When there is already a Package Server available, there are no real performance efficiencies gained on a Local Area Network by enabling multicast. An environment should be examined thoroughly for network topology, package download frequency, and general package size before multicast is enabled on the network. Test 4 Multicast Communications with Varying Package Sizes 1 The purpose of this test is to analyze the traffic patterns and performance of downloading software packages when the package size is varied. In this scenario, a set of 18 managed workstations needed to download a single software package that did not already exist on the client network. In other words, no Package Servers were present, and no other managed workstation already had the software package cached locally on its hard drive. In this test, five (5) different packages were created on the Notification Server. The requisite Software Delivery Tasks were created to deliver the package to a managed workstation. Each task was enabled in turn such that only one task was active on the wire. Only after one task had completed would the next task be enabled. The packages being delivered to the workstation were very simple. The packages themselves consisted of a single file. The command line for the packages was the opening of an Explorer browser on the managed machine. This provided for the creation of packages that could be easily re-usable and distributed. For the purposes of this test, the sizes of the packages were roughly 10K, 100K, 1MB, 10MB, and 50MB. No settings on the Altiris Agent were modified from the baseline to complete this set of tests. Expected Outcome Two separate behavior patterns are expected in the results of this test. The first behavior pattern is for the 10K and 100K packages. Because the minimum package size for multicast communications is 512K, these two packages should be downloaded from the Notification Server by every managed workstation on the client network using unicast TCP/IP sessions. As a result, the time to download for any given workstation should be relatively short. The total time to distribute the package to all managed workstations requesting the package should be no longer than the client configuration interval of 15 minutes. The second behavior pattern deals with the packages of 1M and larger. These packages are subject to multicast transmission because of their size. The time to download for any given Software Package Multicast Performance Testing> 23

24 workstation should be in the range of 8-12 minutes based upon the results of the previous test (Test 3). The total time to distribute the package to all managed workstations requesting the package should be in the range of 20 minutes due to the multicast negotiation windows. For the packages larger than the 512K cutoff, there should only be one (1) download of the package from the Notification Server across the router using a unicast TCP/IP session. The remaining workstations should download the package from the master for the multicast session. Test Results 10K Package Test Table 12 documents the results for the 10K package test. Each machine s time to download is listed along with the average download time and total time for package distribution to all workstations needing the package. Machine Name Time to Obtain Package Information Time to Download DSR02C s 0.19s DSR02C s 0.15s DSR02C s 0.14s DSR02C s 0.34s DSR02C s 0.15s DSR02C s 0.13s DSR02C s 0.13s DSR02C s 0.14s DSR02C s 0.12s DSR02C s 0.14s DSR02C s 0.13s DSR02C s 0.15s DSR03C s 0.18s DSR03C s 0.18s DSR03C s 0.26s DSR03C s 0.15s DSR03C s 0.14s DSR03C s 0.14s Averages 5.76s 0.16s Total Time to Distribute s (8m 54.76s) Table 12: : 18 Machines Downloading a 10K Package As expected, all sessions were unicast transmissions between the managed workstations and the Notification Server. Every machine also performed a speed performance test with the Notification Server as part of obtaining the package information. This resulted in longer preparation times than 24 < Software Package Multicast Performance Testing

25 the baseline times observed in Test 1. However, the time to actually download the package is comparable to the baseline. 100K Package Test Table 13 documents the results for the 100K package test. Each machine s time to download is listed along with the average download time and total time for package distribution to all workstations needing the package. Machine Name Time to Obtain Package Information Time to Download DSR02C s 0.28s DSR02C s 0.24s DSR02C s 0.21s DSR02C s 0.19s DSR02C s 0.20s DSR02C s 0.21s DSR02C s 0.20s DSR02C s 0.21s DSR02C s 0.21s DSR02C s 0.21s DSR02C s 0.19s DSR02C s 0.23s DSR03C s 0.24s DSR03C s 0.25s DSR03C s 0.26s DSR03C s 0.26s DSR03C s 0.28s DSR03C s 0.21s Averages 1.52s 0.22s Total Time to Distribute s (14m 44.33s) Table 13: : 18 Machines Downloading a 100K Package As expected, all sessions were unicast transmissions between the managed workstations and the Notification Server. Because every workstation had recently downloaded a package from the Notification Server no speed performance test was necessary, which reduced the time required in obtaining the package information. Both the time to obtain the package and the time to download the package are in line with the results found in the baseline for a 100K package in Test 1. 1M Package Test Table 14 documents the results for the 1M package test. Because multicast was used to download the package, the table for the data is going to be different from the 10K and 100K results. Software Package Multicast Performance Testing> 25