Spirent Journal of Cloud Application and Security Services PASS Test Methodologies. June 2011 Edition. February 2011 Edition PASS

Transcription

1 Spirent Journal of Cloud Application and Security Services PASS Test Methodologies June 2011 Edition February 2011 Edition PASS

2 Introduction Today s Devices Under Test (DUT) represent complex, multi-protocol network elements with an emphasis on Quality of Service (QoS) and Quality of Experience (QoE) that scale to terabits of bandwidth across the switch fabric. The Spirent Catalogue of Test Methodologies represents an element of the Spirent test ecosystem that helps answer the most critical Performance, Availability, Security and Scale Tests (PASS) test cases. The Spirent Test ecosystem and Spirent Catalogue of Test Methodologies are intended to help development engineers and product verification engineers to rapidly develop and test complex test scenarios. How to use this Journal This provides test engineers with a battery of test cases for the Spirent Test Ecosystem. The journal is divided into sections by technology. Each test case has a unique Test Case ID (Ex. TC_MBH_001) that is universally unique across the ecosystem. Tester Requirements To determine the true capabilities and limitations of a DUT, the tests in this journal require a test tool that can measure router performance under realistic Internet conditions. It must be able to simultaneously generate wire-speed traffic, emulate the requisite protocols, and make real-time comparative performance measurements. High port density for cost-effective performance and stress testing is important to fully load switching fabrics and determine device and network scalability limits. In addition to these features, some tests require more advanced capabilities, such as Integrated traffic, routing, and MPLS protocols (e.g., BGP, OSPF, IS-IS, RSVP-TE, LDP/CR-LDP) to advertise route topologies for large simulated networks with LSP tunnels while simultaneously sending traffic over those tunnels. Further, the tester should emulate the interrelationships between protocol s through a topology. Emulation of service protocols (e.g., IGMPv3, PIM-SM, MP-iBGP) with diminution. Correct single-pass testing with measurement of 41+ metrics per pass of a packet. Tunneling protocol emulation (L2TP) and protocol stacking. True stateful layer 2-7 traffic. Ability to over-subscribe traffic dynamically and observe the effects. Finally, the tester should provide conformance test suites for ensuring protocol conformance and interoperability, and automated applications for rapidly executing the test cases in this journal. Further Resources Additional resources are available on our website at 1

3 Table of Contents Testing Cloud Application and Security Services...3 CAS_001 Determine IP stateful traffic throughput per RFC CAS_002 Stateful traffic concurrent TCP connection capacity per RFC CAS_003 Determine the maximum TCP connection establishment rate CAS_004 Determine the maximum TCP connection tear down rate CAS_005 Measure the BBT external to external basic throughput CAS_006 Measure the BBT external to internal (virtual) max throughput CAS_007 Measure the BBT internal (virtual) to internal (virtual) maximum throughput. 25 CAS_008 Measure performance of HTTP over VPLS over BGP in the cloud CAS_009 BBT External to internal (virtual): scalability and application availability CAS_010 CAS_011 BBT Internal (virtual) to internal (virtual): scalability and application availability in a virtual environment BBT Internal (virtual) to internal (virtual): security attacking the virtual environment CAS_012 BBT External to internal (virtual): security combined traffic and attacks CAS_013 CAS_014 CAS_015 Determine the effects of a denial of service attack on firewall performance per RFC 3511 Section Determine HTTP transfer rate of the firewall performance per RFC3511 Section Measure HTTP transaction rate of the firewall performance per RFC3511 Section CAS_016 Determine the capacity of NAT444 in the cloud CAS_017 Evaluate DNS A and AAAA record performance under attack Appendix A Telecommunications Definitions Appendix B Stateful Playlist by QoS Appendix C MPEG 2/4 Video QoE

4 Testing Cloud Application and Security Services Cloud computing is a general term for anything that involves delivering hosted services over the Internet. These services are broadly divided into three categories: Infrastructure-as-a-Service (IaaS), Platform-as-a- Service (PaaS) and Software-as-a-Service (SaaS). A cloud service has three distinct characteristics that differentiate it from traditional hosting. It is sold on demand, typically by the minute or the hour. It is elastic a user can have as much or as little of a service as they want at any given time. The service is fully managed by the provider. The consumer needs nothing but a personal computer and Internet access. Significant innovations in virtualization and distributed computing, improved access to high-speed Internet, and financial pressure have accelerated interest in cloud computing. IaaS provides virtual server instances with unique IP addresses and blocks of storage on demand. Customers use the provider's application program interface (API) to start, stop, access and configure virtual servers and storage. In the enterprise, cloud computing allows a company to pay for only as much capacity as is needed and bring more online as soon as required. Because this pay-for-what-you-use model resembles the way electricity, fuel and water are consumed, it's sometimes referred to as utility computing. PaaS in the cloud is defined as a set of software and product development tools hosted on the provider's infrastructure. Developers create applications on the provider's platform over the Internet. PaaS providers may use APIs, website portals or gateway software installed on the customer's computer. Currently there are no standards for interoperability or data portability in the cloud. Some providers will not allow software created by their customers to be moved off the provider's platform. 3

5 CAS_001 Determine IP stateful traffic throughput per RFC 3511 Abstract This test determines the throughput of network-layer data traversing the firewall (DUT), as defined in RFC This test determines the maximum throughput for the DUT per fixed packet size. Suggested packet sizes are 64, 128, 256, 512, 768, 1024, 1218, Packets should be UDP packets. Each test should run for at least 10 minutes. References: RFC 3511, RFC 2889, RFC For product verification and engineering. Description The tester offers unicast IP packets to the DUT at a constant rate. The test may consist of either bi-directional or unidirectional traffic. For example, an emulated client may offer a unicast stream of packets to an emulated server, or the tester may simulate a client/server exchange by offering bidirectional traffic. This test employs an iterative search algorithm. For iteration the tester varies the intended load until the maximum rate, at which no packet loss occurs, is found. Since backpressure mechanisms may be employed, resulting in a difference between the intended load and offered load, the test should be performed in either a packet-based or time-based manner. As with RFC 1242, the term packet is used in place of frame. For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: 4 Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. Version 1.0

6 Test Category PASS Testing Stateful Multiplay Benchmark Standards. [x] Performance [ ] Availability [ ] Security [ ] Scale Required Tester Capabilities Topology The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source. Test Procedure 5 1. Reserve two test ports. 2. Connect tester port 1 as the client side to the configured firewall-capable DUT. Connect port 2 as the server side to the other side of the DUT. Establish the link. 3. Begin Step 1 Configure the bandwidth performance test. a. Enter the management IP address of the equipment that is used for this test for the client port. b. Enter the management IP address of the equipment that is used for this test for the server port. c. Select the performance test for specific equipment that is used in this test. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET Per SimUser. iii. Maximum connections per server 1. iv. Maximum transaction per connection 10 or 50. v. User think time 30 seconds. b. Server side: i. HTTP version 1.1.

7 ii. Maximum requests per connection 64 or 10 (if client is set to more than 10). iii. Server type Irrelevant (only changes HTTP headers). iv. Packet size 64b, 128b, 256b, 512b, 768b, 1024b, 1218b, 1514b. 5. End. Variables & Variable Packet sizes 64b, 128b, 256b, 512b, 768b, 1024b, 1218b, 1514b Test Duration These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size Every test duration runs for at least 10 minutes Desired Result For throughput, maximum offered load, expressed in either bits per second or packets per second, at which no packet loss is detected. For forwarding rate, expressed in either bits per second or packet per second, the device is observed to successfully forward to the correct destination interface in response to a specified offered load. Key Measured Metrics Statistic Step Iteration Pass/Fail Status Total packets offered Total packets forwarded Intended load Offered load (if applicable) Forwarding rate Show how many steps are added in the load profile to achieve maximum throughput Show successful/unsuccessful transactions Show total packets offered Show total packets forwarded Show attempted transactions N/A Show forwarding rate Analysis The user should not see lost packets. The user should see the maximum throughput when running the test for each different packet size. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same. 6

8 CAS_002 Stateful traffic concurrent TCP connection capacity per RFC 3511 Abstract This test determines the maximum number of concurrent TCP connections supported through or with the firewall (DUT). This test also finds the maximum number of entries the DUT can store in its connection table. For this test, HTTP 1.1 must be used on both the client and server side. In the case of a proxy-based firewall, the clients must request at least two objects one to measure the TCP establishment time (Time to SYN/ACK and Time to First Byte), the subsequent objects to evaluate the quality of the connection (Average URL Response Time). References: RFC For product verification and engineering. Description This test employs an iterative search algorithm to determine the maximum number of concurrent TCP connections supported through or with the DUT. For each iteration, the aggregate number of concurrent TCP connections attempted by the virtual clients is varied. The destination address is that of the server or that of the NAT proxy. The aggregate rate is defined by the connection attempt rate, and is attempted in a round-robin fashion. To validate all connections, the virtual clients request an object using an HTTP 1.1 or higher GET request. The requests are initiated on each connection after all of the TCP connections have been established. When testing proxy-based DUTs, the virtual clients request two objects using HTTP 1.1 or higher GET requests. The first GET request is required for connection time establishment measurements as specified. The second request is used for validation as previously mentioned. When comparing proxy and non-proxy based DUTs, the test is performed in the same manner. Between each iteration, it is recommended that the tester issue a TCP RST referencing each connection attempted for the previous iteration, regardless of whether or not the connection attempt was successful. The tester waits for aging time before continuing to the next iteration. For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: 7 Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure

9 Version warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. 1.0 Test Category PASS Testing Stateful Multiplay Benchmark Standards. [ ] Performance [ ] Availability [ ] Security [x] Scale Required Tester Capabilities Topology The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source. Test Procedure 8 1. Reserve two test ports. 2. Connect tester port 1 as the client side to the configured firewall capable DUT. Connect port 2 as the server side to the other side of DUT. Establish the link. 3. Begin Step 1 Generate the open connections performance test. a. Enter the management IP address of the equipment that is used for this test for the client port. b. Enter the management IP address of the equipment that is used for this test for the server port.

10 c. Select the performance test for the specific equipment that is used in this test. d. The goal is to open a connection, perform a GET, wait 30 seconds (the connection will remain open unless the DUT has a TCP Timeout lower than this but typically devices have at minimum 90 seconds TCP Timeout), perform another GET, wait 30 seconds, and so on until the ten GETs are complete. e. If the maximum transactions per connection is set to 10, the connection is closed by a FIN/ACK. If the maximum transactions per connection is greater than the number of GETs in the action List, the connections is closed by an RST. f. If the server-side maximum requests per connection is equal to the number of GETs in the client action list, and the client profile is set to a maximum request per connection higher than the number of GETs in the client action list, the connection termination is initiated server-side. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET Per SimUser is 10. iii. Maximum connection per server is 1. iv. Maximum transaction per connection is either 10 or 50. v. User think time is 30 seconds. b. Server side: i. HTTP version 1.1. ii. Maximum requests per connection are either 64 or 10 (if client is set to more than 10). iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b. 5. End. Variables & Variable Packet sizes 64, 512, 1024, Test Duration These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size Every test duration runs for at least 10 minutes Desired Result 9 For maximum concurrent connections, the total number of TCP connections open for the last successful iteration performance in the search algorithm. For minimum connection establishment time, the lowest TCP connection establishment time measured. For maximum connection establishment time, the highest TCP connection establishment time measured. For average connection establishment time, the mean of all measurements of connections establishment times. For aggregate connection establishment time, the total of all measurements of connections establishment time.

11 Key Measured Metrics Statistic Object size Number of completed requests Number of completed responses Show size of object for each test iteration Show number of completed transactions (client side) Show number of completed transactions (server side) Analysis The user should not see lost packets. The user should see the maximum concurrent connections, minimum connection establishment time, maximum connection establishment time, average connection establishment time, aggregate connection establishment time, number of objects requested, and number of objects returned for each of the test iteration that uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same. 10

12 CAS_003 Determine the maximum TCP connection establishment rate Abstract This test determines the maximum TCP connection establishment rate through or with the firewall (DUT). This test also finds the maximum rate at which the DUT can update its connection table. The test defines the aggregate number of TCP connections that must be established. HTTP 1.1 or higher must be used for this test for both clients and servers. The client and server must use the same HTTP version. The test also defines the number of bytes, excluding any bytes associated with the HTTP header, to be transferred in response to an HTTP 1.1 or higher GET request. References: RFC 3511, RFC For product verification and engineering. Description This test employs an iterative search algorithm to determine the maximum rate at which the DUT can accept TCP connection requests. For each iteration, the aggregate rate at which TCP connection requests are attempted by the virtual clients is varied. The destination address is that of the server or that of the NAT proxy. The aggregate number of connections, defined by number of connections, is attempted in a round-robin fashion. The same application-layer object transfers required for validation and establishment time measurements as described in the concurrent TCP connection capacity test is performed. Between each iteration, it is recommended that the tester issue a TCP RST referencing each connection attempted for the previous iteration, regardless of whether or not the connection attempt was successful. The tester waits for aging time before continuing to the next iteration. For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: 11 Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business.

13 Version 1.0 Test Category PASS Testing Stateful Multiplay Benchmark Standards. [ ] Performance [ ] Availability [ ] Security [x] Scale Required Tester Capabilities Topology Specific Spirent features that enable this test case and emphasize the Spirent differentiators and branding. Test Procedure Reserve two test ports. 2. Connect tester port 1 as the client side to the configured firewall capable DUT. Connect port 2 as the server side to the other side of the DUT. Establish the link. 3. Begin Step 1 Configure the connections per second performance test. a. Enter the management IP address of the equipment that is used for this test for the client port. b. Enter the management IP address of the equipment that is used for this test for the server port. c. Select the performance test for the specific equipment that is used in this test. d. Make sure the test is set up to use HTTP 1.1 for both client and server.

14 e. To maximize the performance of the tester, this test sends 10 HTTP level 1 GET requests from each SimUser. f. Each TCP connection accepts one HTTP transaction as the maximum transaction, therefore each of the SimUsers generates 10 TCP connections sequentially. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET per SimUser is 10. iii. A maximum connection per server is 1. iv. Maximum transaction per connection is 1. v. User think time 30 seconds. b. Server side: i. HTTP version 1.1. ii. A maximum request per connection is 1. iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b. 5. End. Variables & Variable Packet sizes 64b, 512b, 1024b, 10240b Test Duration These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size Every test duration runs for at least 10 minutes Desired Result For highest connection rate, in connections per second, the rate for which all connections successfully opened in the search algorithm. For minimum connection establishment time, the lowest TCP connection establishment time measured. For maximum connection establishment time, the highest TCP connection establishment time measured. For average connection establishment time, the mean of all measurements of connection establishment times. For aggregate connection establishment time, the total of all measurements of connection establishment times. Key Measured Metrics Statistic Number of object requested Number of object returned Show the total number requested objects Show the total number of returned objects 13

15 Analysis The user should not see lost packets. The user should see the maximum TCP connections rate, minimum connection establishment time, maximum connection establishment time, average connection establishment time, aggregate connection establishment time, number of object requested, and number of object returned for each of the test iteration that uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same. 14

16 CAS_004 Determine the maximum TCP connection tear down rate Abstract This test determines the maximum TCP connection teardown rate through or with the firewall (DUT). This test defines the number of TCP connections that are attempted to be torn down. It also defines a method for closing TCP connections. The test must be performed with either a three-way or four-way handshake. In a four-way handshake, each side sends separate FIN and ACK messages. In a three-way handshake, one side sends a combined FIN/ACK message upon receipt of a FIN. The test also defines whether closing of connections is to be initiated from the client or from the server. References: RFC 3511, RFC For product verification and engineering. Description This test employs an iterative search algorithm to determine the maximum TCP connection tear down rate supported by the DUT. The test iterates through different TCP connection tear down rates with a fixed number of TCP connections. In the case of proxy based DUTs, the DUT receives the ACK in response to issuing a FIN packet to close its side of the TCP connection. For validation purposes, the virtual client or server, whichever is applicable, may verify that the DUT received the final ACK by re-transmitting the final ACK. A TCP RST should be received in response to the retransmitted ACK. Between each iteration, it is RECOMMENDED that the virtual clients or servers, whichever is applicable, issue a TCP RST referencing each connection which was attempted to be torn down, regardless of whether or not the connection tear down attempt was successful. The test waits for aging time before continuing to the next iteration. For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: 15 Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure.

17 Version Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. 1.0 Test Category PASS Testing Stateful Multiplay Benchmark Standards. [ ] Performance [ ] Availability [ ] Security [x] Scale Required Tester Capabilities Topology The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source. Test Procedure Reserve two test ports. 2. Connect tester port 1 as the client side to the configured firewall-capable DUT. Connect port 2 as the server side to the other side of the DUT. Establish the link. 3. Begin Step 1 Generate the connections per second performance test.

18 a. Enter the management IP address of the equipment that is used for this test for the client port. b. Enter the management IP address of the equipment that is used for this test for the server port. c. Select the performance test for the specific equipment that is used in this test. d. Make sure the test is set up to use HTTP 1.1 for both client and server. e. To maximize the performance of the tester, this test sends 10 HTTP level 1 GET requests from each SimUser. f. Each TCP connection accepts one HTTP transaction as the maximum transaction, therefore each of the SimUser will generate 10 TCP connections sequentially. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET per SimUser is 10. iii. Maximum connections per server is 1. iv. Maximum transaction per connection is 1. v. User think time 30 seconds. b. Server side: i. HTTP version 1.1. ii. Maximum requests per connection is 1. iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b. 5. End. Variables & Variable Packet sizes 64b, 512b, 1024b, 10240b Test Duration These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size Every test duration runs for at least 10 minutes Desired Result For highest connection tear down rate, the highest rate in connections per second, for which all TCP connections were successfully torn down in the search algorithm. For minimum connection tear down time, the lowest TCP connection tear down time measured. For maximum connection tear down, the highest TCP connection tear down time measured. For average connection tear down time, the mean of all measurements of connection tear down time. For aggregate connection tear down time, the total of all measurements of connection tear down time. 17

19 Key Measured Metrics Statistic Number of connections Aging time Close method Close direction Show the total number of connections Show the aging time Show 3-way or 4-way handshake Show close of connections are initiated from either client or server Analysis The user should not see lost packets. The user should see the maximum TCP connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time, number of connections, aging time, close method, and close direction for each of the test iteration which uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same. 18

20 CAS_005 Measure the BBT external to external basic throughput Abstract This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines a baseline performance including the maximum realistic throughput of the DUT under sustained attack. For product verification and engineering. Description This test determines maximum throughput. The DUT is connected to the tester via a 10 GbE connection and also connected to the main test network via 1 Gbps connections to the virtual elements of the test bed. Measures throughput of the external-to-external throughput of a DUT Measures performance of virtualized applications Version 1.0 Test Category Testing Security and Cloud Services. PASS [x] Performance [ ] Availability [ ] Security [ ] Scale Required Tester Capabilities Ability to import the BBT test file Support for server transactions profiles for different packet sizes Real-time and summary reports on packet sizes, test duration, and throughput. 19

21 Topology Test Procedure 1. Reserve two test ports. 2. Connect tester port 1 as the client side to the DUT. Connect port 2 as the server side to the other side of DUT. Establish the link. 3. Begin Step 1 Import the test file (spf) from BBT a. Select the Ext_to_Ext_Final_PlusAttacks_0001 test from SBFinal Project b. Set the client side port to the network that is currently connected. c. Set the server side port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET per SimUser. iii. Maximum connections per server 2. iv. Maximum requests per connection 50. v. User thinks time 0 seconds. vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec). b. Server side: i. HTTP version 1.1. ii. Maximum requests per connection 64. iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 100KB. 5. Begin Step 3 Run the test 6. End. 20

22 Variables & Variable Packet sizes 100KB Stateful Threats (Attack List) 16 threats were used Test Duration This packet size is intended to determine the maximum throughput for the DUT There are 16 stateful threats that are used to determine whether the DUT is able to block them and still sustain the throughput. Each test duration runs for at least 12 minutes. Desired Result For throughput, the maximum offered load, at which no packet loss is detected. Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Exploitation Results Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput Show successful/unsuccessful transactions Show attempted transactions Show attack type, client success exploit ratio Show throughput level of DUT Analysis The user should not see lost packets. The user should see the maximum throughput when running the test using 100KB objects and while blocking threat traffic. 21

23 CAS_006 Measure the BBT external to internal (virtual) max throughput Abstract This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines the performance from external (physical) to internal (virtual) environments including the maximum realistic throughput of the DUT. For product verification and engineering. Description This test determines the maximum throughput when generating traffic to move from the physical to the virtual elements of the test bed. Determines the maximum virtual max throughput. Version 1.0 Test Category Testing Security and Cloud Services. PASS [ ] Performance [x] Availability [ ] Security [ ] Scale Required Tester Capabilities Ability to import the BBT test file Support for server transactions profiles for different packet sizes Real-time and summary reports on packet sizes, test duration, and throughput 22

24 Topology Test Procedure Reserve three test ports (1 client and 2 servers). 2. Connect the physical tester to the DUT. Run the virtual tester in a VM on the DUT. Establish the link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Ext_to_Int_Bandwidth_1200b_Post test from SBFinal Project. b. Set the client port to the network that is currently connected. c. Set the server ports port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET and POST per SimUser. iii. Maximum connections per server 2. iv. Maximum requests per connection 50. v. User think time 0 seconds. vi. Global load profile for http traffic (SimUsers/sec). b. Server side: i. HTTP version 1.1. ii. Maximum requests per connection 64. iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 1200 bytes. 5. Begin Step 3 Run the test. 6. End.

25 Variables & Variable Packet sizes for GET 500, 1024, 1200 bytes Packet sizes for POST 500, 1024, 1200 bytes This packet size is used to determine the maximum throughput for the DUT. This packet size is used to determine the maximum throughput for the DUT. Desired Result For throughput, the maximum offered load, at which no packet loss is detected. Sample results below: External To Internal Tests GET GET GET POST POST POST Packet size (bytes) Incoming (kbps) Outgoing (kbps) Transactions Per Second Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput Show successful/unsuccessful transactions Show attempted transactions Show throughput level of DUT Analysis In this test, examine the packet size for both GET and POST methods. Compare the measured results to the desired results in the above table. Report the results as a table comparing the differences. 24

26 CAS_007 Measure the BBT internal (virtual) to internal (virtual) maximum throughput Abstract This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines the performance from internal (virtual) to internal (virtual) environments including the maximum realistic throughput of the DUT. For product verification and engineering. Description This test determines the maximum throughput when generating traffic to move from the internal (virtual) to the virtual elements of test bed. This test determines whether the virtual controller in the DUT affects performance, by generating HTTP traffic to achieve the maximum level of throughput in the virtual environments. Measures the virtual to virtual maximum performance Version 1.0 Test Category Testing Security and Cloud Services. PASS [x] Performance [ ] Availability [ ] Security [ ] Scale Required Tester Capabilities Ability to import the BBT test file Support for server transactions profiles for different packet sizes Real-time and summary reports on packet sizes, test duration, and throughput 25

27 Topology Test Procedure Reserve three test ports (1 client and 2 servers). 2. Connect the physical tester to the DUT. Run the virtual tester in a VM on the DUT. Establish the link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Int_to_Int test from SBFinal Project. b. Set the client port to the network that is currently connected. c. Set the server ports to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET per SimUser. iii. Maximum connections per server 2. iv. Maximum requests per connection 50. v. User think time 0 seconds. vi. Global load profile for http traffic (SimUsers/sec). b. Server side: i. HTTP version 1.1. ii. Maximum requests per connection 64. iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 100KB. 5. Begin Step 3 Run the test. a. Run with the virtual controller redirector in the DUT disabled. Traffic will pass straightthrough. b. Run with the virtual controller redirector in the DUT enabled. The DUT will read every packet as it passes through. 6. End.

28 Variables & Variable Packet sizes for GET 100KB, 500 bytes, 1024 bytes, and 1200 bytes Test Duration This packet size is used to determine the maximum throughput for the DUT. Each test duration runs for at least 12 minutes. Desired Result For throughput, the maximum offered load, at which no packet loss is detected. Sample results below: External To Internal Tests GET GET GET Packet size (bytes) Incoming (kbps) Outgoing (kbps) Transactions Per Second Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Throughput Results (Incoming and Outgoing Traffic) Transactions per second Show how many steps are added in the load profile to achieve maximum throughput Show successful/unsuccessful transactions Show attempted transactions Show throughput level of DUT Show how many transactions per second are successfully for different packet size Analysis The user should not see lost packets. The user should see the maximum throughput for each test iteration. When testing with different packet sizes, the user should see different throughput measurements and the DUT configuration must remain the same. The user should see performance improve significantly with a higher performance server. 27

29 CAS_008 Measure performance of HTTP over VPLS over BGP in the cloud Abstract Description Version HTTP goodput is a critical core metric when analyzing a service provider core network. This test allows the user to measure HTTP goodput ranges over a discrete number of VPLS tunnels. Without understanding the range of goodput across the cloud, the service provider will not be able to properly design and deploy the proper network scale of each device in the network. References: RFC2616 Hyper Text Markup Languages. For functional and system performance testing. HTTP goodput is the measure of delivered bandwidth to the HTTP stack with all network impairments taken into consideration. Since traffic in the service provider cloud is tunneled and backhauled through the cloud, impairments in the core can have a direct impact on the overall user experience of web based traffic, end to end. Impairments such as network induced jitter, excessive latency distribution, sequencing errors, and errors can all contribute to poor network traffic. Understanding goodput ranges in a specific network environment is a key attribute of end user experience. 1.0 Test Category PASS Performance Test. [x] Performance [ ] Availability [ ] Security [ ] Scale Required Tester Capabilities The test equipment should have the ability the ability to dynamically setup control plane routing and BGP and VPLS. Over VPLS, the user should have the ability to make dynamic changes to HTTP traffic. The tester should have the ability to coordinate and trend responses. 28

30 Topology Test Procedure 1. Reserve tester ports A and B. connect the ports to the DUT and bring up the links. 2. If P router emulation is selected, setup desired number of P routers per port. a. Establish one EBGP peering session per P router to the DUT. 3. Setup the desired number of PE router per tester port or emulated P router. a. Emulate 1 CE router per PE router. b. Setup a host clouds behind each CE router. 4. Advertise though BGP all host cloud subnets. 5. Establish one LDP VPLS tunnel from each CE emulated port on tester A to each emulated CE router on tester port B. Verify all tunnels establish. 6. On tester port B, set up HTTP Servers with the desired HTTP return object size. Start all HTTP servers. 7. On the tester port A, setup one HTTP client per host. Each client should be connected to a unique emulated HTTP server. 8. For each client, set the following load specifications: a. Ramp from 0 kbps to (desired peak aggregate bandwidth / tester port A HTTP host count) in 30 seconds. b. Set steady state for (total test time 100 seconds). c. Ramp down to 0 kbps in 30 seconds. 9. Start all HTTP clients. 10. Measure aggregate and per host goodput. 29

31 Variables & Variable Emulate P Routers? Number of P Routers Per Port PE Routers or PE Routers per Port? Tester Port A Starting BGP AS Tester Port A Starting IP Address Tester Port A Netmask Tester Port A Starting AS Tester Port A Starting BGP AS Tester Port A DUT IP Address Tester Port B Starting IP Address Tester Port B Netmask Tester Port B Starting AS Tester Port B DUT IP Address Hosts per CE Cloud HTTP Object Size Test Duration Peak Aggregate HTTP Bandwidth Ask user is they wish to Emulate P Routers or just PE routers. Default=Yes How many P Routers to Emulate per Port. Default=5 Defines how many PE Routers to emulate per tester port or per P Router Starting AS Number of Tester Port A Starting IP Address of the First P or PE Router Classless Netmask of Tester Port A Starting AS of the P or PE Router Starting AS Number of Tester Port A Tester Port B DUT IP Address Starting IP Address of the First P or PE Router Classless Netmask of Tester Port A Starting AS of the P or PE Router Tester Port B DUT IP Address Number of Hosts per CE Cloud. Default=5 Object Size to Return. Default=384 bytes Duration of the test. Default=600 Seconds Peak Aggregate HTTP Bandwidth. Default=150 Mbps Desired Result The DUT should be able to deliver the offered bandwidth. The variance in goodput per individual host should be zero. Key Measured Metrics Statistic Aggregate HTTP Goodput Per Host HTTP Goodput Total HTTP achieved globally Per Host HTTP goodput Analysis To facilitate analysis, the following tables and charts should be created. The offered and measured bandwidth and goodput should be displayed as a bar chart comparison, there should also be a table presenting direct numbers and difference in kbps and percentage. The next chart should be offered bandwidth vs. aggregate goodput every 10 seconds. 30 There should be a chart showing per host goodput with the X-Axis representing VPLS tunnel and associated transactions behind each tunnel, and the Y-Axis repressing measured goodput as a percentage of offered bandwidth.

32 CAS_009 BBT External to internal (virtual): scalability and application availability Abstract This test is delivered by Broadband Testing (BBT) and focuses on securing the virtual data center. This test case determines application availability under load from external to internal (virtual) environments and scalability of server performance. Description This test determines application availability when generating traffic to move from external to virtual elements of the BBT test bed. This test also determines scalability of server performance and response times while under high load. Performance (outright throughput and application availability/latency) Scalability Security Efficiency Version 1.0 Test Category Testing Security and Cloud Services PASS [x] Performance [x] Availability [x] Security [x] Scale Required Tester Capabilities Import BBT test file Server transactions profile support for different packet sizes Real-time and summary reports packet sizes, test duration, and server response time. 31

33 Topology Test Procedure Reserve three test ports (1 client and 2 servers) 2. Connect cabling to the DUT. Connect an Avalanche 3100 and an Avalanche Virtual (running on a VM) as the physical and virtual components. Also connect to a management server (which consists of an IPS appliance, software running on VMs under VMware, vcontroller, SMS management server, and management software). Establish link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Ext_to_Int_CC and Ext_to_Int_CPS tests from SBFinal Project. b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser. iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50. v. User Think Time 0 seconds. vi. Global load profile for http traffic (SimUsers and cps). b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection 64. iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 500, 1024, and 1200 bytes. 5. Begin Step 3 Running the test. 6. End.

34 Variables & Variable Packet sizes for GET 500, 1024, 1200 bytes Packet sizes for POST 500, 1024, 1200 bytes Test Duration This packet size determines the maximum throughput for the DUT. This packet size determines the maximum throughput for the DUT. Every test duration runs for at least 12 minutes Desired Result The result of this external-to-internal environment test demonstrates that the firewall is able to scale with more powerful servers in sub milliseconds using different packet sizes for virtual machines and while sustaining maximum throughput and application availability under load. External To Internal Tests GET GET GET POST POST POST Packet size (bytes) Server Response Time (ms) Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Server Response Time (ms) Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput. Show successful/unsuccessful transactions. Show attempted transactions. Show server response time. Show throughput level of DUT. Analysis The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput measurements. 33

35 CAS_010 BBT Internal (virtual) to internal (virtual): scalability and application availability in a virtual environment Abstract This test is delivered by Broadband Testing (BBT) and focuses on securing the virtual data center. This test case determines application availability under load from internal (virtual) to internal (virtual) environments and scalability of server performance. Description This test determines application availability when generating traffic to move from internal to virtual elements of the BBT test bed. This test also determines the scalability of server performance and response time while under high load. Performance (outright throughput and application availability/latency) Scalability Security Efficiency Version 1.0 Test Category Testing Security and Cloud Services PASS [x] Performance [x] Availability [x] Security [x] Scale Required Tester Capabilities Import BBT test file Server transactions profile support for different packet sizes Real-time and summary reports packet sizes, test duration, and server response time. 34

36 Topology Test Procedure Reserve three test ports (1 client and 2 servers) 2. Connect cabling to the DUT. Connect Avalanche virtual (running on a VM) as the virtual components to management server (which consists of IPS appliance, software running on VMs under VMware, vcontroller, SMS management server, and management software). Establish link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Int_to_Int_Bandwidth_1200b_0001, Int_to_Int_Bandwidth_1k_0001, and Int_to_Int_Bandwidth_500_0001 tests from SBFinal Project b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser. iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50. v. User Think Time 0 seconds. vi. Global load profile for http traffic (SimUsers and cps). b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection 64. iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 500, 1024, and 1200 bytes. 5. Begin Step 3 Running the test. 6. End.

37 Variables & Variable Packet sizes for GET 500, 1024, 1200 bytes Test Duration This packet size determines the maximum throughput for the DUT. Every test duration runs for at least 12 minutes Desired Result The result of this external-to-internal environment test demonstrates that the firewall is able to scale with more powerful servers in sub milliseconds using different packet sizes for virtual machines and while sustaining maximum throughput and application availability under load. External To Internal Tests GET GET GET Packet size (bytes) Server Response Time (ms) Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Server Response Time (ms) Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput. Show successful/unsuccessful transactions. Show attempted transactions. Show server response time. Show throughput level of DUT. Analysis The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput measurements. 36

38 CAS_011 BBT Internal (virtual) to internal (virtual): security attacking the virtual environment Abstract This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center. This test case determines performance, including the maximum realistic throughput in the virtual environment, from internal (virtual) to internal (virtual) while under sustained attack. Description This test determines the maximum throughput within the virtual environment while under a series of different attacks. This test uses only the internal (virtual) environment and is setup using Avalanche virtual targeting vcontroller directly with a combination of http traffic load generation and injected threat traffic, first based on CodeRed II followed by several different stateful attacks. Performance (outright throughput and application availability/latency) Scalability Security Efficiency Version 1.0 Test Category Testing Security and Cloud Services PASS [x] Performance [x] Availability [x] Security [x] Scale Required Tester Capabilities Import BBT test file Server transactions profile support for different packet sizes Real-time and summary reports packet sizes, test duration, and server response time. 37

39 Topology Test Procedure Reserve two test ports. 2. Connect cabling to the DUT. Connect vcontroller and management server and Avalanche virtual (running on a VM) as the virtual and virtual component (which consists of IPS appliance, software running on VMs under VMware, vcontroller, SMS management server, and management software). Establish link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Int_to_Int_Final_PlusAttacks_10G test from SBFinal Project. b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser. iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50. v. User Think Time 0 seconds. vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec) b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection 64. iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 100KB. 5. Begin Step 3 Running the test. a. Generate HTTP traffic over 90 seconds, then inject 1432 CodeRed II attacks. b. Repeat the same test, generate HTTP traffic over 90 seconds, then inject multiple stateful attacks.

40 c. Repeat the same test, generate HTTP traffic over 10 minutes, then inject multiple stateful attacks for 80 seconds into the test. d. Repeat the same test, generate HTTP traffic over 10 minutes, then inject multiple stateful attacks for 80 seconds into the test. On the IPS DUT, add a rule to the firewall to block a batch of clients based on IP address. 6. End. Variables & Variable Packet sizes 100KB Stateful Threats (Attack List) Test Duration This packet size determines the maximum throughput for the DUT Stateful threats Every test duration runs for at least 90 seconds or 10 minutes Desired Result The first test generates the maximum throughput of HTTP traffic over 90 seconds runs and attacks are injected, the DUT in the virtual environment successfully blocks the bad traffic with no reduction in overall maximum throughput. The second test generates the maximum throughput of HTTP traffic for over 90 seconds and multiple stateful attacks are injected which were contained within the signature database, the DUT in the virtual environment successfully block all the bad traffic. Since the attacks attack across a saturated link VM to VM the result of this test should show the vcontroller s ability to successfully block with no reduction in overall maximum throughput. The third test should generate the maximum throughput of HTTP traffic and the DUT should block all bad traffic. Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Server Response Time (ms) Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput. Show successful/unsuccessful transactions. Show attempted transactions. Show server response time. Show throughput level of DUT. Analysis 39 The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput measurements

41 CAS_012 BBT External to internal (virtual): security combined traffic and attacks Abstract This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center. This test case combines 10Gbps traffic from Avalanche 3100 (external) to the firewall and 1 Gbps traffic within the virtual environment on the vcontroller to determine a baseline performance including the maximum realistic throughput of the DUT under sustained attack. Description This test determines the maximum throughput from an external to a virtual environment. The DUT is connected to a Spirent 3100 via a 10GbE connection and is also connected to the main test network via 1 Gbps connections to the virtual elements of the test bed with multiple stateful threats on the vcontroller. The test bed is designed to run at 1 GbE and 10GbE (in the physical space) test options to emulate a physical or virtual data center environment. Performance (outright throughput and application availability/latency) Scalability Security Efficiency Version 1.0 Test Category Testing Security and Cloud Services PASS [x] Performance [x] Availability [x] Security [x] Scale Required Tester Capabilities Import BBT test file Server transactions profile support for different packet sizes Real-time and summary reports packet sizes, test duration, and server response time. 40

42 Topology Test Procedure Reserve two test ports. 2. Connect cabling to the DUT. Connect Avalanche GigE directly to the Firewall DUT and Avalanche virtual 1GigE (running on a VM) as the physical and virtual components, also connect to management server (which consists of IPS appliance, software running on VMs under VMware, vcontroller, SMS management server, and management software). Establish link. 3. Begin Step 1 Import the test file (spf) from BBT. a. Select the Threats_test test from SBFinal Project. b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 Test the effect of load, packet size, and duration. a. Client side: i. HTTP version 1.1 ii. HTTP GET Per SimUser. iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50. v. User Think Time 0 seconds. vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec) b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection 64. iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 1024 bytes. 5. Begin Step 3 Running the test. a. Generate HTTP traffic between Avalanche G to IPS DUT then inject multiple stateful attacks directly at the vcontroller. 6. End.

43 Variables & Variable Packet sizes 1024 bytes Stateful Threats (Attack List) 16 threats Test Duration This packet size determines the maximum throughput for the DUT There are 16 stateful threats to determine whether the DUT is able to block them and still sustain the throughput Every test duration runs for at least 10 minutes Desired Result For throughput, maximum offered load, at which no packet loss is detected. The external and virtual environment generates around 4.5 Gbps of maximum throughput between the Avalanche GigE and the firewall DUT and multiple stateful threats are injected at the vcontroller. The vcontroller should be able to sustain maximum throughput while blocking threat traffic in addition to http traffic sending 1024 bytes of GET requests. Key Measured Metrics Statistic Step Iteration Pass/Fail Status Intended load Server Response Time (ms) Throughput Results (Incoming and Outgoing Traffic) Show how many steps are added in the load profile to achieve maximum throughput. Show successful/unsuccessful transactions. Show attempted transactions. Show server response time. Show throughput level of DUT. Analysis The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput measurements. 42

44 CAS_013 Determine the effects of a denial of service attack on firewall performance per RFC 3511 Section 5.5 Abstract This test determines the effect of a denial of service attack on a firewall (DUT) TCP connection establishment and/or HTTP transfer rate. The denial of service handling test must be run after obtaining baseline measurements. The TCP SYN flood attack exploits TCP s three-way handshake mechanism by having an attacking source host generate TCP SYN packets with random source addresses towards a victim host, thereby consuming that host s resources. Without determining the performance of the firewall under attack, the user will not understand the corner case traffic conditions of the DUT. Description Use the same procedure as defined in the Maximum TCP Connection Establishment Rate test case depending on whether testing against the baseline TCP connection establishment rate. In addition, the test instrument generates TCP SYN packets targeting the server s IP address or NAT proxy address at a rate defined by the SYN attack rate. The test instrument originating the TCP SYN attack must be attached to the unprotected network. In addition, the test instrument must not respond to the SYN/ACK packets sent by the target server or NAT proxy in response to the SYN packet. Target Users Product verification, Engineering Target Device Under Test (DUT) Firewall Reference RFC For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing

45 valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. Version 1.0 Test Category Testing Firewalls and VPN PASS [X] Performance [X] Availability [X] Security [X] Scale Required Tester Capabilities Generate performance test for connections per second. Load profiles for different step iterations and test duration. Server transactions profile support for different object sizes. Real-time and summary reports on highest connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time. 44

46 Topology Diagram Test Procedure Reserve two test ports. 2. Connect cabling to the DUT. Cable tester port 1 as the client side to the configured firewallcapable DUT. Connect port 2 as the server side to the other side of DUT. Establish the link. 3. Begin Step 1 Generate the connections per second performance test. a. Enter the management IP address of the client port. b. Enter the management IP address of the server port. c. Select the performance test for the specific equipment used in this test. d. Configure HTTP 1.1 for both client and server. e. To maximize the performance of the tester, send 10 HTTP level 1 GET requests from each SimUser. f. Each TCP connection accepts one HTTP transaction as maximum transaction, therefore each of the SimUsers generate 10 TCP connections sequentially. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET Per SimUser is 10. iii. Maximum Connections Per Server is 1. iv. Maximum Transaction per Connection is 1. v. User Think Time 30 seconds. b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection is 1.

47 iii. Server Type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1024b, 10240b. 5. Begin Step 3 Run test. a. Generate HTTP traffic over 90 seconds then inject CodeRedII attacks. b. Repeat the same test, generating HTTP traffic over 90 seconds then inject multiple stateful attacks. c. Repeat the same test, generating HTTP traffic over 10 minutes then inject multiple stateful attacks for 80 seconds into the test. d. Repeat the same test, generating HTTP traffic over 10 minutes then inject multiple stateful attacks for 80 seconds into the test. On the IPS DUT, add a rule to the firewall to block a batch of clients based on IP address. 6. End test. Control Variables & Variable Default Value Packet sizes 64b, Determine the maximum throughput for the firewall 64 bytes 512b, 1024b, 10240b DUT per fixed packet size. Test Duration At least 10 minutes. 10 minutes Key Measured Metrics Matric Metric Unit Step Iteration Indicates how many steps are added in the load profile to achieve maximum throughput. Pass/Fail Status Indicates successful/unsuccessful transactions. Intended load Indicates attempted transactions. Exploitation Results Indicates attack type, client success exploit ratio. Throughput Results (Incoming and Outgoing Traffic) Indicates throughput level of DUT. Desired Result In the first test the DUT in the virtual environment should successfully block the bad traffic with no reduction in overall maximum throughput. In the second test, the DUT should successfully block all the bad traffic. Since the attacks attacking across a saturated link, this test should show the ability of DUT to successfully block with no reduction in overall maximum throughput. 46

48 Analysis In the third test, the DUT should able to block all bad traffic. Exploitation Results Attack Client/Server Attack Name Attack Type Client Success Client Failure Server Success antvillexss.xml coderedii.xml core_mailslot_dos.xml ms xml WebDAV_rs.xml Stateful Stateful Stateful Stateful Stateful No packets should be lost. Maximum throughput should be achieved when running the test using 512Kb, 1024Kb, 1Mb objects and while blocking threat traffic. 47

49 CAS_014 Determine HTTP transfer rate of the firewall performance per RFC3511 Section 5.6 Abstract This test determines the transfer rate of HTTP requested objects traversing the firewall. It defines the aggregate number of connections attempted. The number should be a multiple of the number of virtual clients participating in the test. It also defines the method of closing TCP connections. This test must be performed with either a three-way or four-way handshake. In a four-way handshake, each side sends separate FIN and ACK messages. In a three-way handshake, one side sends a combined FIN/ACK message upon receipt of a FIN. Finally, it defines whether closing of connections is to be initiated from the client or from the server. With the test, the user can determine the peak HTTP rate of the DUT. Description Each HTTP 1.1 or higher virtual client requests one or more objects from an HTTP 1.1 or higher server using one or more HTTP GET requests over each connection. The aggregate number of connections attempted, defined by number of connections, must be evenly divided among all the participating virtual clients. If the virtual clients make multiple HTTP GET requests per connection, it must request the same object size for each GET request. Multiple iterations of this test may be run with objects of different sizes. Target Users Product verification, Engineering Target Device Under Test (DUT) Firewall Reference RFC For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful.

50 Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. Version 1.0 Test Category Testing Firewalls and VPN PASS [X] Performance [X] Availability [X] Security [X] Scale Required Tester Capabilities Generate performance test for connections per second. Load profiles for different step iterations and test duration. Server transactions profile support for different object sizes. Real-time and summary reports on highest connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time. 49

51 Topology Diagram Test Procedure Reserve two test ports. 2. Connect cabling to the DUT. Cable tester port 1 as the client side to the configured firewallcapable DUT. Connect port 2 as the server side to the other side of DUT. Establish the link. 3. Begin Step 1 Generate the HTTP 1.1 transactions per second performance test: a. Enter the management IP address of the client port. b. Enter the management IP address of the server port. c. Select the performance test for the specific equipment used in this test. d. Configure HTTP 1.1 for both client and server. e. To maximize the performance of the tester, send 10 HTTP level 1 GET requests from each SimUser. f. Each TCP connection accepts one HTTP transaction as maximum transaction, therefore each of the SimUsers generate 1 TCP connection sequentially. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET Per SimUser is 50. iii. Maximum Connections Per Server is 1. iv. Maximum Transaction per Connection is 50. v. User Think Time 0 seconds. b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection is 10.

52 iii. Connection Termination with FIN. iv. Object/Page size is 100Kb, 500Kb, 1Mb. 5. Load Reference. a. Load (Transactions/second) 1150 with 100Kb object. b. Load (Transactions/second) 230 with 500Kb object. c. Load (Transactions/second) 110 with 1Mb object. 6. End test. Control Variables & Variable Default Value Packet sizes 100Kb, 500Kb, 1Mb Determine the maximum throughput for the firewall DUT per fixed packet size. 980 Mbps throughput Test Duration At least 10 minutes. 10 minutes Key Measured Metrics Matric Metric Unit Number of bytes Total payload bytes transferred transferred Number of Timeouts Total number of timeout events Retransmitted bytes Total number of retransmitted bytes Desired Result Analysis The transfer rate results are reported for each of the object sizes tested, including the number and percentage of completed requests, number and percentage of completed responses, and the resultant transfer rate for each iteration of the test. The transfer rate results also include the total bytes transferred, total timeouts, total retransmitted bytes and resultant goodput. There should be no packet loss. Metrics of interest include maximum TCP connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time, number of connections, aging time, close method, and close direction. Different packet sizes should produce different throughput measurements with the same firewall DUT configuration. 51

53 CAS_015 Measure HTTP transaction rate of the firewall performance per RFC3511 Section 5.7 Abstract This test determines the maximum transaction rate the firewall can sustain. It defines method for closing TCP connections. The test must be performed with either a three-way or four-way handshake. In a four-way handshake, each side sends separate FIN and ACK messages. In a threeway handshake, one side sends a combined FIN/ACK message upon receipt of a FIN. It defines whether closing of connections is to be initiated from the client or from the server. This test ensures the users understand the upper transaction rate limit of the firewall to prevent anomalous dropped packets and sessions. Description For each iteration of the test, HTTP 1.1 or higher simulated clients vary the aggregate GET request rate offered to HTTP 1.1 or higher servers. The simulated clients maintain the offered request rate for the defined test duration. If a simulated client makes multiple HTTP GET requests per connection, it must request the same object size for each GET request. Multiple tests may be performed with different object sizes. Target Users Product verification, Engineering Target Device Under Test (DUT) Firewall Reference RFC For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls: Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful. Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure

54 warning. Second, the failure state of the firewall is known firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure. Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business. Version 1.0 Test Category Testing Firewalls and VPN PASS [X] Performance [X] Availability [X] Security [X] Scale Required Tester Capabilities Generate performance test for connections per second. Load profiles for different step iterations and test duration. Server transactions profile support for different object sizes. Real-time and summary reports on highest connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time. 53

55 Topology Diagram Test Procedure Reserve two test ports. 2. Connect cabling to the DUT. Cable tester port 1 as the client side to the configured firewallcapable DUT. Connect port 2 as the server side to the other side of DUT. Establish the link. 3. Begin Step 1 Generate the HTTP 1.1 transactions per second performance test: a. Enter the management IP address of the client port. b. Enter the management IP address of the server port. c. Select the performance test for the specific equipment used in this test. d. Configure HTTP 1.1 for both client and server. e. To maximize the performance of the tester, send 10 HTTP level 1 GET requests from each SimUser. f. Each TCP connection accepts one HTTP transaction as maximum transaction, therefore each of the SimUsers generate 10 TCP connections sequentially. 4. Begin Step 2 Test the effect of load, object size, and duration. a. Client side: i. HTTP version 1.1. ii. HTTP GET Per SimUser is 50. iii. Maximum Connections Per Server is 1. iv. Maximum Transaction per Connection is 50. v. User Think Time 0 seconds. b. Server side: i. HTTP version 1.1. ii. Maximum Requests per Connection is 10.

56 iii. Connection Termination with FIN. iv. Object/Page size is 64b, 512b, 1024b, 10240b. 5. End test. Control Variables & Variable Default Value Packet sizes 64b, 512b, 1024b, 10240b Determine the maximum throughput for the firewall DUT per fixed packet size. 980 Mbps throughput Test Duration At least 10 minutes. 10 minutes Key Measured Metrics Matric Metric Unit Maximum Maximum rate for all transactions, that is all tps transaction rate requests/responses are completed Max transaction Transaction time starts when client issues GET request ms time and end when client receives the last bit of request object Min transaction Transaction time starts when client issues GET request ms time and end when client receives the last bit of request object Average transaction time Transaction time starts when client issues GET request and end when client receives the last bit of request object ms Desired Result Analysis The test results include GET request attempt rate, number of requested attempted, number and percentage of requests completed, number of responses attempted, number and percentage of responses completed minimum transaction time, average transaction time, and maximum transaction time. The test results also include the number of connections attempted, number and percentage of connections completed, number and percentage of connection teardowns completed. There should be no packet loss. Metrics of interest include maximum TCP connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time, number of connections, aging time, close method, and close direction. Different packet sizes should produce different throughput measurements with the same firewall DUT configuration. 55

57 CAS_016 Determine the capacity of NAT444 in the cloud Abstract Description NAT444, or Carrier Grade NAT, provides a mechanism to extend the IPv4 address space within a carrier core. This test case determines the peak capacity of a Carrier Grade NAT device. Frequently NAT444 is used in a transition strategy from IPv4 to mixed IPv4 and IPv6 networks. Determining the upper scale of the NAT Device under Test (DUT) allows for proper capacity planning in the network. As IPv4 addresses allocated to carriers by IANA run out, carriers need ways to better utilize their existing allocation of IPv4 addresses. One technique is NAT444, or industrial grade NAT. Effectively, NAT444 is a double network address translation. The subscriber s cloud is translated by the default gateway router. (This is sometimes called NAT44.) The upstream translated traffic is further translated by an industrial grade NAT router. This technique has the potential for problems. Specifically, the technique breaks the OSI model, adds single points of failure, and can cause performance and protocol problems for subscribers. This test measures the peak capacity of a NAT444 industrial grade router. The following QoS table details the pass/fail targets: Codepoint Max Jitter (usec) Max Latency (usec) Max Loss (Frames) Max Duplicate (Frames) Max Reordered (Frames) Max Late (Frames) EF 0 >= AF AF AF BE ANY ANY ANY ANY ANY ANY Target Users Product Verification, Engineering, End users Target Device Under Test (DUT) Type Device Under Test (DUT) is an industrial grade NAT444 Router Reference 56 draft-shirasaki-nat444-isp-shared-addr-05 RFC 4787 RFC 5382 RFC 5508

58 Version Measuring the peak performance of a NAT444 router is critical to the deployment capacity of NAT in the network. With the potential for so many performance issues, NAT444 must be carefully tuned and measured before deployment. Because significant modification to the subscriber traffic occurs, there is a large potential for jitter, sequencing, latency, and corruption issues. Special consideration for loss, duplication, re-order, out of order, and late packets must be taken into account. In addition, a NAT44 device can be very susceptible to microburst conditions and real-time change. 1.0 Test Category PASS CAS [x] Performance [ ] Availability [ ] Security [ ] Scale Required Tester Capabilities To test NAT444 fully, the tester must be able to measure latency, jitter, loss, re-order, duplication, late, and error is one pass of the packet. In addition the tester must have the ability to make real-time changes to content, add and subtract content, and produce microbursts. Topology Diagram 57

59 Test Procedure 1. Begin a. Setup a subscriber pool of dual-homed IPv4 and IPv6 subscribers. Set 255 hosts per client subscriber. Randomize the IPv4 destination. Set the IPv4 source address to the address of the Public NAT address. Encapsulate IPv6 using 6over4 rules. b. On the IPv6 hosts, setup HTTP, SIP, and Video. c. Transmit traffic through the NAT444 DUT. d. Setup the number of subscribers as measured in the performance test. 2. Loop until a QoS or QoE failure is detected. a. Continuously measure the CPU of the DUT using Spirent itest (1 Sample every 1 Second). b. Take down 50% of the subscribers, wait 30 seconds. c. Measure the QoE of the remaining subscribers while bringing the downed 50% of subscribers back up. d. Take down a random number of subscribers for 30 seconds. e. Measure the QoE of the remaining subscribers while bringing the downed random number of subscribers back up. f. Examine the variance of the CPU utilization sample. The variance should be no more than +/- 5%. a. if no failures are measured, add 10% more subscribers, otherwise stop. 3. End Loop Control Variables & Variable Default Value Duration Time per Iteration 120 seconds Number of Subscribers Pressure on the NAT444 DUT 1000 Random Subscribers Number of Subscribers to Down/Up 50% Key Measured Metrics Metric Metric Unit HTTP Goodput Ratio QoE Metric ratio for Web traffic (Sent : Measured) >0.99 SIP MOS Measured SIP Quality >4.5 Subscriber capacity Measured number of Peak Subscribers Subscribers Video MOS-AV Measure Video Quality >4.5 Desired Result As the number of subscribers increase, decrease and change, the QOS/QoE metrics of traffic should be both consistent and high. There should be no measured packet errors. 58

60 Analysis When examining QoE metrics, pay special attention to when traffic is changing in the network. Specifically, when subscriber traffic is being added in microburst, you should see almost no change to the QoS and QoE metrics during the time of bursts and change in general. In addition, pay special attention to QoS foundation variables like packet reorder and duplication. These are signs that you are approaching the upper limit of the NAT444 DUT. 59

61 CAS_017 Evaluate DNS A and AAAA record performance under attack Abstract DNS protocols are a key determining factor of overall user experience. This test measures the ability of DNS servers to respond in a predictable manner under extreme load. This test allows the user to determine the scale to which DNS may be provisioned in the network. Description Domain Name System (or Service or Server) is an Internet service that translates domain names into IP addresses. Because domain names are alphabetic, they're easier to remember than IP addresses. The Internet however, is really based on IP addresses. Every time you use a domain name, therefore, a DNS service must translate the name into the corresponding IP address. For example, the domain name might translate to The DNS system is, in fact, its own network. If one DNS server doesn't know how to translate a particular domain name, it asks another one, and so on, until the correct IP address is returned. This test measures the capacity of DNS A Record and DNSv6 AAAA Record requests against a live set of the top internet sites. Target Users Enterprise, PV, Engineering Target Device Under Test (DUT) Primary and Secondary DNS Servers Reference RFC 1034 RFC 3901 DNS quality of experience and stability is a critical component of a user s perception of network performance. The predictability of DNS address resolution has a direct correlation to perceived performance. In addition, it is assumed that DNS must perform even while under attack. Version Test Category CAS

62 PASS [x] Performance [ ] Availability [x] Security [ ] Scale Required Tester Capabilities The tester must have the ability to generate predictable A and AAAA record rates while simultaneously attacking the primary and secondary DNS servers Topology Diagram Test Procedure 1. Reserve the test port and bring up link. 2. Determine the desired rates of a list for A Record and AAA record look ups. 3. Program the list of the top 500 Internet sites into the tester. 4. Determine the baseline by testing the top 120 sites at 1 request per second. Determine the maximum resolve time, as defined by the latency between the sent request and the reception of the resolve. Measure the maximum resolve time for both A Record and AAAA Record. 5. Sync the Attack database with spirent.com. Program all DNS attacks in the database at 100 attacks per second, targeting both the primary and secondary DNS servers. 6. Start the attacks For each of the desired A and AAAA Record Rates: a. Set the rate for both A and AAAA requests. b. Round Robin the list of DNS sites to be resolved

63 c. Set the desired duration. Set a mix of 90% DNS requests to the primary DNS and 10% to the secondary DNS. d. Transmit DNS requests. e. Measure the maximum resolve time and the number of failed resolves. f. Repeat for the desired number of trials. 8. Stop the attacks. 9. End. Control Variables & Variable Default Value Primary DNS Server DNS Server NA Secondary DNS Server Backup DNS Server NA Top 500 Sites List of top 500 DNS Resolves List A Record Rate List List of DNS Resolve Rates per Second 1,1000, 5000, Iteration Duration Time per Iteration 120 Seconds Trials Trails per Rate 3 Key Measured Metrics Metric Metric Unit A Record Resolve Latency Maximum time to resolve A Record msec AAAA Record Resolve Latency Maximum time to resolve A Record msec Desired Result Analysis The baseline of 1 resolve per second for A Record and AAAA Record should be 2 msec or faster. The variance for additional rate and trials should be no slower than 1% of the baseline. Report the maximum time per rate per trial in the form of a table and chart, one table and chart for A Record resolves and one for AAAA Record resolves. 62

64 CAS_018 Virtualized SQL Server Network Footprint Testing Abstract Test multiple configurations of multi-tiered applications using an Oracle SQL backend: small, medium, and large. Description Target Users This test determines and quantifies the impact of moving to a virtual SQL Server and the number of virtual SQL servers that one hypervisor platform can support. The PASS/fail criteria shows successful/unsuccessful transactions. Physical Server resources ( CPU, Memory ) SAN-Based Storage Virtualization Platform Cloud Test Engineers Target Device Under Test (DUT) Virtualized cloud of Exchange Servers and clusters Reference TBD Exchange Server is a critical business process. It is important to know how it performs when virtualized. Version 1.0 Test Category VCE PASS [ X ] Performance [ X ] Availability [ ] Security [ X ] Scale Required Tester Capabilities 63 Ability to run in a virtualized environment as a simulated Exchange Server from the point of view of protocols ( MAPI, POP3, SMTP ) and also the ability to connect over these same protocols to a real Exchange Server.

65 Test Procedure 1. Connect the Avalanche 3100B to the network portion of the vblock. 2. The Avalanche 3100B emulates clients performing HTTP forms database request to SQL servers running on the UCS. 3. The SQL servers run on a VM on the UCS. a. The SQL servers can be real servers or be emulated by running Avalanche Virtual on the UCS. b. Multiple instances of real or emulated servers may run on the UCS. 4. The Avalanche Clients access data stored within the vblock through these servers. 5. Install and configure one or more SQL Servers on the UCS. 6. The Server can be real, emulated or a mix of both, depending on the use case. 7. An Avalanche 3100B set up outside the vblock emulates the Clients. These Clients request data stored within the vblock through the servers. 8. Scale the number of Clients and Servers to stress the resources (memory and CPU) of the vblock. a. Scale either the Clients or the Servers or both at the same time. You may configure multiple clients per server. 9. You may configure background traffic for additional load outside and within the vblock to cause more strain on the vblock. Control Variables & Variable Default Value Transfer Size 100% 8KB ; 100% 2KB Random I/O Percent 100% Read I/O Percent 67% Disk Capacity Full Users per CPU One (1) Quantity of Outstanding I/O Varied Key Measured Metrics Statistic Metric Unit Number of the SQL Clients and Servers that the vblock can Success Ordinal Numeric support without a major impact on performance. The response time of the vblock for the Clients requesting the data. The performance of the vblock for this test in presence of background/additional traffic and load Throughput and other Benchmark criteria of the vblock in the presence of this load Desired Result 64 Analysis Success with no failed transactions Compare this to previous results obtained from a physical exchange server.

66 VCE_0019 Virtualized Exchange Server Footprint Testing Abstract Test multiple configurations of Exchange: small, medium, and large, in a virtual platform. Determine the performance and availability and scale. Description Target Users This test determines whether to install physical or virtual Exchange Server deployments based on the performance of Exchange server in a physical versus a virtualized environment. The PASS/fail criteria is successful/unsuccessful transactions. Physical Server resources ( CPU, Memory ) SAN-Based Storage Virtualization Platform Cloud Test Engineers Target Device Under Test (DUT) Virtualized cloud of Exchange Servers and clusters Reference TBD Exchange server is a critical business process. It is important to know how it performs when virtualized. Version 1.0 Test Category VCE PASS [ X ] Performance [ X ] Availability [ ] Security [ X ] Scale Required Tester Capabilities 65 Ability to run in a virtualized environment as a simulated Exchange Server from the point of view of protocols ( MAPI, POP3, SMTP ) and also the ability to connect over these same protocols to a real Exchange Server.

67 Topology Diagram Test Procedure 1. Connect an Avalanche 3100B to the Network portion of the vblock. 2. The Avalanche 3100B emulates Exchange clients requesting data stored on the vblock. 3. The Exchange Server runs on a VM on the UCS. It can be a real server or emulated by running Avalanche Virtual on the UCS. Multiple instances of real or emulated servers may be run on the UCS. The Avalanche Clients access data stored within the vblock through these servers. 4. Install and configure one or more Exchange Servers on the UCS. The servers can be real, emulated or a mix of both, depending on the use case. 5. An Avalanche 3100B setup outside the vblock emulates the Clients, which perform MAPI request for data stored within the vblock through the servers. 6. Scale the number of Clients and Servers to stress the resources (memory an CPU) of the vblock. Scale either the clients or the servers or both at the same time. You may configure multiple clients per server. 7. You may configure background traffic for additional load outside and within the vblock to cause more strain on the vblock. Control Variables & 66 Variable Default Value Transfer Size 100% 8KB ; 100% 2KB Random I/O Percent 100% Read I/O Percent 67% Disk Capacity Full Users per CPU One (1) Quantity of Outstanding I/O Varied

68 Key Measured Metrics Statistic Metric Unit Client/Server request/response Success Ordinal Numeric Total number of the Exchange Clients and Servers that the vblock can support without a major impact on performance Response time of the vblock in the presence of background traffic Overall Goodput Connections/second Desired Result Analysis Success with no failed transactions Intended load - Show attempted transactions. Intended load - Show attempted transactions Intended load - Show attempted transactions. You need to compare this to previous results obtained from a physical exchange server. 67

69 VCE_020 Virtualized CIFS/NFS Network Footprint Testing Abstract Test various CIFS and NFS operations. Description Target Users This test determines and quantifies the impact of moving to a virtual SQL Server and the number of virtual SQL servers that can be supported on one Hypervisor platform. The PASS/fail criteria are successful/unsuccessful transactions. Physical Server resources ( CPU, Memory ) SAN-Based Storage Virtualization Platform Cloud Test Engineers Target Device Under Test (DUT) Virtualized cloud of SQL Servers and clusters Reference TBD SQL Server is a critical business process. It is important to know how it performs when virtualized. Version 1.0 Test Category VCE PASS [ X ] Performance [ X ] Availability [ ] Security [ X ] Scale Required Tester Capabilities 68 Ability to run in a virtualized environment as a simulated SQL Server from the point of view of protocols ( MAPI, POP3, SMTP ) and also the ability to connect over these same protocols to a real SQL Server.

70 Test Procedure 1. Connect an Avalanche 3100B to the network portion of the vblock. 2. The Avalanche 3100B emulates clients performing CIFS/NFS or a mix of both requests to the UCS through the Servers. 3. Configure each VM on the UCS as a Linux server running either CIFS or NFS or both. The Avalanche Clients access data stored within the vblock through these servers. 4. Install and configure one or more Linux servers on the UCS. The server needs to run SAMBA/NFS or both the protocols. 5. An Avalanche 3100B setup outside the vblock emulates the Clients, which request data stored within the vblock through the servers. 6. Scale the number of Clients and Servers to stress the resources (memory and CPU) of the vblock. You may scale either the clients, the servers or both at the same time. You may have multiple clients per server. 7. You may configure background traffic for additional load outside and within the vblock to cause more strain on the vblock. Control Variables & Variable Default Value Transfer Size 100% 8KB ; 100% 2KB Random I/O Percent 100% Read I/O Percent 67% Disk Capacity Full Users per CPU One (1) Quantity of Outstanding I/O Varied Key Measured Metrics Statistic Metric Unit Number of Clients and Servers that the vblock can support without a major impact on performance The response time of the vblock for the Clients requesting the data The performance of the vblock for this test in presence of background/additional traffic and load Throughput and other Benchmark criteria of the vblock in the presence of this load Desired Result Analysis Success with no failed transactions. Compare this to previous results obtained from a physical Exchange Server. 69

71 VCE_021 Virtualized Web Server Network Footprint Testing Abstract Simulate http traffic for IIS and Apache. Description Target Users UCS running Instances of IIS/Apache/Avalanche HTTP/HTTPS Servers or mix of each Cloud Test Engineers Target Device Under Test (DUT) Reference Virtualized cloud of web servers and clusters TBD Version Web servers are a critical business process. It is important to know how it performs when virtualized. 1.0 Test Category PASS VCE [ X ] Performance [ X ] Availability [ ] Security [ X ] Scale Required Tester Capabilities Ability to run in a virtualized environment as a simulated web server from the point of view of protocols ( MAPI, POP3, SMTP ) and also the ability to connect over these same protocols to a real web server. Test Procedure Connect an Avalanche 3100B to the network portion of the vblock. 2. The Avalanche 3100B emulates clients performing TCP/HTTP/HTTPS requests or a mix of these requests to the UCS through the servers. 3. Configure each VM on the UCS as IIS/Apache/Avalanche HTTP/HTTPS. The Avalanche Clients access data stored within the vblock through these servers. a. The VM may run a real IIS or Apache Server.

72 b. A real HTTP/HTTPS server may run Avalanche Virtual on the VM. c. A mix of both [a] and [b] can be configured to simulate a more realistic environment. 4. Install and configure one or more servers on the UCS. 5. An Avalanche 3100B setup outside the vblock emulates the Clients, which request data stored within the vblock through the servers. 6. Scale the number of Clients and Servers to stress the resources (memory and CPU) of the vblock. You may scale either the clients or the servers or both at the same time. You may have multiple clients per server. 7. You may configure background traffic for additional load outside and within the vblock to cause more strain on the vblock. Control Variables & Variable Default Value Transfer Size 100% 8KB ; 100% 2KB Random I/O Percent 100% Read I/O Percent 67% Disk Capacity Full Users per CPU One (1) Quantity of Outstanding I/O Varied Key Measured Metrics Statistic Metric Unit Number of Clients and Servers that the vblock can support without a major impact on performance Success Ordinal Numeric Response time of the vblock for the Clients requesting the data Performance of the vblock for this test in presence of background/additional traffic and load Throughput and other Benchmark criteria of the vblock in the presence of this load Desired Result Analysis Success with no failed transactions. Compare this to previous results obtained from a physical server. 71

73 Appendix A Telecommunications Definitions APPLICATION LOGIC. The computational aspects of an application, including a list of instructions that tells a software application how to operate. APPLICATION SERVICE PROVIDER (ASP). An ASP deploys hosts and manages access to a packaged application by multiple parties from a centrally managed facility. The applications are delivered over networks on a subscription basis. This delivery model speeds implementation, minimizes the expenses and risks incurred across the application life cycle, and overcomes the chronic shortage of qualified technical personnel available in-house. APPLICATION MAINTENANCE OUTSOURCING PROVIDER. Manages a proprietary or packaged application from either the customer's or the provider's site. ASP INFRASTRUCTURE PROVIDER (AIP). A hosting provider that offers a full set of infrastructure services for hosting online applications. ATM. Asynchronous Transport Mode. An information transfer standard for routing high-speed, highbandwidth traffic such as real-time voice and video, as well as general data bits. AVAILABILITY. The portion of time that a system can be used for productive work, expressed as a percentage. BACKBONE. A centralized high-speed network that interconnects smaller, independent networks. BANDWIDTH. The number of bits of information that can move through a communications medium in a given amount of time; the capacity of a telecommunications circuit/network to carry voice, data, and video information. Typically measured in Kbps and Mbps. Bandwidth from public networks is typically available to business and residential end-users in increments from 56 Kbps to 45 Mbps. BIT ERROR RATE. The number of transmitted bits expected to be corrupted per second when two computers have been communicating for a given length of time. BURST INFORMATION RATE (BIR). The rate of information in bits per second that the customer may need over and above the CIR. A burst is typically a short duration transmission that can relieve momentary congestion in the LAN or provide additional throughput for interactive data applications. BUSINESS ASP. Provides prepackaged application services in volume to the general business market, typically targeting small to medium size enterprises. 72 BUSINESS-CRITICAL APPLICATION. The vital software needed to run a business, whether custom-written or commercially packaged, such as accounting/finance, ERP, manufacturing, human resources and sales databases.

74 BUSINESS SERVICE PROVIDER. Provides online services aided by brick-and-mortar resources, such as payroll processing and employee benefits administration, printing, distribution or maintenance services. The category includes business process outsourcing (BPO) companies. COMMERCE NETWORK PROVIDER. Commerce networks were traditionally proprietary value-added networks (VANs) used for electronic data interchange (EDI) between companies. Today the category includes the new generation of electronic purchasing and trading networks. COMPETITIVE ACCESS PROVIDER (CAP). A telecommunications company that provides an alternative to a LEC for local transport and special access telecommunications services. CAPACITY. The ability for a network to provide sufficient transmitting capabilities among its available transmission media, and respond to customer demand for communications transport, especially at peak usage times. CLIENT/DEVICE. Hardware that retrieves information from a server. CLUSTERING. A group of independent systems working together as a single system. Clustering technology allows groups of servers to access a single disk array containing applications and data. COMPUTING UTILITY PROVIDER (CUP). A provider that delivers computing resources, such as storage, database or systems management, on a pay-as-you-go basis. CSU/DSU. Channel Server Unit/Digital Server Unit. A device used to terminate a telephone company connection and prepare data for a router interface. DATA MART. A subset of a data warehouse, intended for use by a single department or function. DATA WAREHOUSE. A database containing copious amounts of information, organized to aid decisionmaking in an organization. Data warehouses receive batch updates and are configured for fast online queries to produce succinct summaries of data. DEDICATED LINE. A point-to-point, hardwired connection between two service locations. DEMARCATION LINE. The point at which the local operating company's responsibility for the local loop ends. Beyond the demarcation point (also known as the network interface), the customer is responsible for installing and maintaining all equipment and wiring. DISCARD ELIGIBILITY (DE) BIT. Relevant in situations of high congestion, it indicates that the frame should be discarded in preference to frames without the DE bit set. The DE bit may be set by the network or by the user; and once set cannot be reset by the network. DS-1 OR T-1. A data communication circuit capable of transmitting data at 1.5 Mbps. Currently in widespread use by medium and large businesses for video, voice, and data applications. 73 DS-3 OR T-3. A data communications circuit capable of transmitting data at 45 Mbps. The equivalent data capacity of 28 T-1s. Currently used only by businesses/institutions and carriers for high-end applications.

75 ELECTRONIC DATA INTERCHANGE (EDI). The electronic communication of business transactions (orders, confirmations, invoices etc.) of organizations with differing platforms. Third parties provide EDI services that enable the connection of organizations with incompatible equipment. ENTERPRISE ASP. An ASP that delivers a select range of high-end business applications, supported by a significant degree of custom configuration and service. ENTERPRISE RELATIONSHIP MANAGEMENT (ERM). Solutions that enable the enterprise to share comprehensive, up-to-date customer, product, competitor and market information to achieve long-term customer satisfaction, increased revenues, and higher profitability. ENTERPRISE RESOURCE PLANNING (ERP). An information system or process integrating all manufacturing and related applications for an entire enterprise. ERP systems permit organizations to manage resources across the enterprise and completely integrate manufacturing systems. ETHERNET. A local area network used to connect computers, printers, workstations, and other devices within the same building. Ethernet operates over twisted wire and coaxial cable. EXTENDED SUPERFRAME FORMAT. A T1 format that provides a method for easily retrieving diagnostics information. FAT CLIENT. A computer that includes an operating system, RAM, ROM, a powerful processor and a wide range of installed applications that can execute either on the desktop or on the server to which it is connected. Fat clients can operate in a server-based computing environment or in a stand-alone fashion. FAULT TOLERANCE. A design method that incorporates redundant system elements to ensure continued systems operation in the event of the failure of any individual element. FDDI. Fiber Distributed Data Interface. A standard for transmitting data on optical-fiber cables at a rate of about 100 Mbps. FRAME. The basic logical unit in which bit-oriented data is transmitted. The frame consists of the data bits surrounded by a flag at each end that indicates the beginning and end of the frame. A primary rate can be thought of as an endless sequence of frames. FRAME RELAY. A high-speed packet switching protocol popular in networks, including WANs, LANs, and LAN-to-LAN connections across long distances. GBPS. Gigabits per second, a measurement of data transmission speed expressed in billions of bits per second. HOSTED OUTSOURCING. Complete outsourcing of a company's information technology applications and associated hardware systems to an ASP. 74 HOSTING PROVIDER. Provider who operates data center facilities for general-purpose server hosting and collocation. INFRASTRUCTURE ISV. And independent software vendor that develops infrastructure software to support the hosting and online delivery of applications.

76 INTEGRATED SERVICES DIGITAL NETWORK (ISDN). An information transfer standard for transmitting digital voice and data over telephone lines at speeds up to 128 Kbps. INTEGRATION. Equipment, systems, or subsystem integration, assembling equipment or networks with a specific function or task. Integration is combining equipment/systems with a common objective, easy monitoring and/or executing commands. It takes three disciplines to execute integration: 1) hardware, 2) software, and 3) connectivity transmission media (data link layer), interfacing components. All three aspects of integration have to be understood to make two or more pieces of equipment or subsystems support the common objective. INTER-EXCHANGE CARRIER (IXC). A telecommunications company that provides telecommunication services between local exchanges on an interstate or intrastate basis. INTERNET SERVICE PROVIDER (ISP). A company that provides access to the Internet for users and businesses. INDEPENDENT SOFTWARE VENDOR (ISV). A company that is not a part of a computer systems manufacturer that develops software applications. INTERNETWORKING. Sharing data and resources from one network to another. IT SERVICE PROVIDER. Traditional IT services businesses, including IT outsourcers, systems integrators, IT consultancies and value added resellers. KILOBITS PER SECOND (KBPS). A data transmission rate of 1,000 bits per second. LEASED LINE. A telecommunications line dedicated to a particular customer along predetermined routers. LOCAL ACCESS TRANSPORT AREA (LATA). One of approximately 164 geographical areas within which local operating companies connect all local calls and route all long-distance calls to the customer's interexchange carrier. LOCAL EXCHANGE CARRIER (LEC). A telecommunications company that provides telecommunication services in a defined geographic area. LOCAL LOOP. The wires that connect an individual subscriber's telephone or data connection to the telephone company central office or other local terminating point. LOCAL/REGIONAL ASP. A company that delivers a range of application services, and often the complete computing needs, of smaller businesses in their local geographic area. MEGABITS PER SECOND (MBPS). 1,024 kilobits per second. METAFRAME. The world's first server-based computing software for Microsoft Windows NT 4.0 Server, Terminal Server Edition multi-user software (co-developed by Citrix). 75 MODEM. A device for converting digital signals to analog and vice versa, for data transmission over an analog telephone line. MULTIPLEXING. The combining of multiple data channels onto a single transmission medium. Sharing a circuit - normally dedicated to a single user - between multiple users.

77 MULTI-USER. The ability for multiple concurrent users to log on and run applications on a single server. NET-BASED ISV. An ISV whose main business is developing software for Internet-based application services. This includes vendors who deliver their own applications online, either directly to users or via other service providers. NETWORK ACCESS POINT (NAP). A location where ISPs exchange traffic. NETWORK COMPUTER (NC). A thin-client hardware device that executes applications locally by downloading them from the network. NCs adhere to a specification jointly developed by Sun, IBM, Oracle, Apple and Netscape. They typically run Java applets within a Java browser, or Java applications within the Java Virtual Machine. NETWORK COMPUTING ARCHITECTURE. A computing architecture in which components are dynamically downloaded from the network onto the client device for execution by the client. The Java programming language is at the core of network computing. ONLINE ANALYTICAL PROCESSING (OLAP). Software that enables decision support via rapid queries to large databases that store corporate data in multidimensional hierarchies and views. OPERATIONAL RESOURCE PROVIDER. Operational resources are external business services that an ASP might use as part of its own infrastructure, such as helpdesk, technical support, financing, or billing and payment collection. OUTSOURCING. The transfer of components or large segments of an organization's internal IT infrastructure, staff, processes or applications to an external resource such as an ASP. PACKAGED SOFTWARE APPLICATION. A computer program developed for sale to consumers or businesses, generally designed to appeal to more than a single customer. While some tailoring of the program may be possible, it is not intended to be custom-designed for each user or organization. PACKET. A bundle of data organized for transmission, containing control information (destination, length, origin, etc.), the data itself, and error detection and correction bits. PACKET SWITCHING. A network in which messages are transmitted as packets over any available route rather than as sequential messages over circuit-switched or dedicated facilities. PEERING. The commercial practice under which nationwide ISPs exchange traffic without the payment of settlement charges. PERFORMANCE. A major factor in determining the overall productivity of a system, performance is primarily tied to availability, throughput and response time. 76 PERMANENT VIRTUAL CIRCUIT (PVC). A PVC connects the customer's port connections, nodes, locations, and branches. All customer ports can be connected, resembling a mesh, but PVCs usually run between the host and branch locations. POINT OF PRESENCE (POP). A telecommunications facility through which the company provides local connectivity to its customers.

78 PORTAL. A company whose primary business is operating a Web destination site, hosting content and applications for access via the Web. REMOTE ACCESS. Connection of a remote computing device via communications lines such as ordinary phone lines or wide area networks to access distant network applications and information. REMOTE PRESENTATION SERVICES PROTOCOL. A set of rules and procedures for exchanging data between computers on a network, enabling the user interface, keystrokes, and mouse movements to be transferred between a server and client. RESELLER/VAR. An intermediary between software and hardware producers and end users. Resellers frequently add value (thus Value-Added Reseller) by performing consulting, system integration and product enhancement. ROUTER. A communications device between networks that determines the best path for optimal performance. Routers are used in complex networks of networks such as enterprise-wide networks and the Internet. SCALABILITY. The ability to expand the number of users or increase the capabilities of a computing solution without making major changes to the systems or application software. SERVER. The computer on a local area network that often acts as a data and application repository and that controls an application's access to workstations, printers and other parts of the network. SERVER-BASED COMPUTING. A server-based approach to delivering business-critical applications to end-user devices, whereby an application's logic executes on the server and only the user interface is transmitted across a network to the client. Benefits include single-point management, universal application access, bandwidth-independent performance, and improved security for business applications. SINGLE-POINT CONTROL. One of the benefits of the ASP model, single-point control helps reduce the total cost of application ownership by enabling widely used applications and data to be deployed, managed and supported at one location. Single-point control enables application installations, updates and additions to be made once, on the server, which are then instantly available to users anywhere. SPECIALIST ASP. Provide applications which serve a specific professional or business activity, such as customer relationship management, human resources or Web site services. SYSTEMS MANUFACTURER. Manufacturer of servers, networking and client devices. TELECOMS PROVIDER. Traditional and new-age telecommunications network providers (telcos). THIN CLIENT. A low-cost computing device that accesses applications and and/or data from a central server over a network. Categories of thin clients include Windows-Based Terminals (WBT, which comprise the largest segment), X-Terminals, and Network Computers (NC). 77 TOTAL COST OF OWNERSHIP (TCO). Model that helps IT professionals understand and manage the budgeted (direct) and unbudgeted (indirect) costs incurred for acquiring, maintaining and using an application or a computing system. TCO normally includes training, upgrades, and administration as well as the purchase price. Lowering TCO through single-point control is a key benefit of server-based computing.

79 TOTAL SECURITY ARCHITECTURE (TSA). A comprehensive, end-to-end architecture that protects the network. TRANSMISSION CONTROL PROTOCOL/INTERNET PROTOCOL (TCP/IP). A suite of network protocols that allow computers with different architectures and operating system software to communicate over the Internet. USER INTERFACE. The part of an application that the end user sees on the screen and works with to operate the application, such as menus, forms and buttons. VERTICAL MARKET ASP. Provides solutions tailored to the needs of a specific industry, such as the healthcare industry. VIRTUAL PRIVATE NETWORK (VPN). A secure, encrypted private connection across a cloud network, such as the Internet. WEB HOSTING. Placing a consumer's or organization's web page or web site on a server that can be accessed via the Internet. WIDE AREA NETWORK. Local area networks linked together across a large geographic area. WINDOWS-BASED TERMINAL (WBT). Thin clients with the lowest cost of ownership, as there are no local applications running on the device. Standards are based on Microsoft's WBT specification developed in conjunction with Wyse Technology, NCD, and other thin client companies. 78

80 Name Appendix B Stateful Playlist by QoS The following patterns represent industry best practices in modeling stateful traffic across a Device Under Test (DUT) by QoS. Enterprise Campus Apps DiffServ EF (Real-Time) VoIP 15% (SIP+RTP+G.729A),U nicast Web Conference 2-Way (MPEG2-TS, VBR), SIP 5% Higher Education Network Administration 2% (SSH) DiffServ 0x31 (Critical) Routing 3% OSPF Routing Updates 2%, BGP Updates 1%), Database 17% (Oracle SQLNet Updates), Corporate Web 2%, IMAP4 5% SQL 7% SQLNet SQL Table Updates), HTTPS University Admin 3% (64 Bytes index.html, 5x 1K JPEG Images), Video Conference 5% (MPEG2TS, VBR, 480i), VoIP 5% (G.729A CODEC) DiffServ 0x20 (General) Multicast Video 13% (480i, MPGEG-2, IGMPv2, 5 Multicast Channels), Telnet/SSH (2%), CIFS 10% (1:1:3 Small/Medium/Large Ratio) FTP 7% (Large Files), HTTPS Student Services, HTTP 3%, POP3/SMTP 9%, CIFS 8% (1:1:3 Small/Medium/Large Objects, bidirectional), Multicast Video 5% (480i) DiffServ 0x00 (Best Effort) Internet Web 5% HTTP (1024 Byte index.html, Byte JPEG, 5 1K JPEG, 1x 100k jpeg), BitTorrent 11% IM 12% (AIM), BitTorrent 24%, HTTP 3% (1024 Byte index.html, Byte JPEG, 5 1K JPEG, 1x 100k jpeg), HTTPS 1% (64 Bytes index.html, 5x 1K JPEG Images), Mail 5%, FTP 1% (Large Files), Telnet/SSH 3% Service Providers Telnet/SSH 1% BGP Route Updates 1% N/A 50% P2P (Bit Torrent, 5% Peer to Tracker, 95% Peer- 2-Peer), 30% HTTP (1024 Byte index.html, Byte JPEG, 5 1K JPEG, 1x 100k jpeg), 5% DNS, Video (MPEG2-TS 5%), SIP (G.729A 3%), Gaming (WoW 5%), 2% RAW TCP 10G Max Bandwidth No Payload, RAW TCP 1G max Bandwidth No Payload, RAW TCP Small/Medium Business Apps POP2/SMTP 15% (5:2:1 Ratio of Small/medium/Big ratio). HTTPS 20% (64 Bytes index.html, 5x 1K JPEG Images, CIFS 30% (1:1:3 Small/Medium/Large Objects, bidirectional, BitTorrent 10%, Internet Web 25% HTTP (1024 Byte index.html, Byte JPEG, 5 1K JPEG, 1x 100k jpeg) WAN Accelerator Network Control 5% CIFS 40% (1:1:3 (Windows Domain Small/Medium/Large Fields). Controller Updates), Exchange 35%(5:2:1 Network Logins Small/Medium/Large ratio) Internet AppMix 2011 HTTPS 10% (64 Bytes index.html, 5x 1K JPEG Images) BitTorrent 10% 50% P2P (Bit Torrent, 5% Peer to Tracker, 95% Peer- 2-Peer), 30% HTTP (1024 Byte index.html, Byte JPEG, 5 1K JPEG, 1x 100k jpeg), 5% DNS, Video (MPEG2-TS 5%), SIP (G.729A 3%), Gaming (WoW 5%), 2% RAW TCP 79

81 Appendix C MPEG 2/4 Video QoE The following information is a typical pattern for MPGE2TS based video streams with a normalized MOS- AV schedule. 80