The impact of multi-core processor on web server performance

Transcription

1 The impact of multi-core processor on web server performance Frane Urem and Želimir Mikulić Department of management College of Šibenik Complete Address: Trg A. Hebranga, Šibenik, 000, Croatia Phone: (+385) Fax: (+385) Abstract : The single processor systems are history and the multi-core and many-core systems world is here. Systems performance scales with number of cores only if systems software and applications are designed to fully exploit the parallelism built in multi-core platforms. In this paper we have tried to answer on question is the web server good platform to exploit advantage of the increased raw compute power that comes with the availability of the added cores. We have used a wellknown queuing theory result to compute the average response time of a request at a web server and compared simple performance model with performance scaling results from tests performed on today s real desktop systems. I. INTRODUCTION The key to web server performance is in the capability to use all possible hardware resources on a single system. Clients communicate with servers through many independent flows (or connections). If web server application processing and the associated network protocol processing of a flow are done exclusively on a single core, we expect minimal data sharing and synchronization between flows. That s why we expect that web server and web application software can use flow-level parallelism to increase throughput with the number of CPU cores. Typical stack of layers that are existing on all web servers is shown on Figure. Hardware Since they respond to mutually independent client requests, they should scale easily with the number of cores by exploiting flow-level parallelism. To test this, we set up test server running a well-tuned IIS 7.0. HTTP server and Windows 008 Server operating system. We have tested server with two and four cores with pairs of cores sharing L cache. The Windows 008 Server kernel supports a parallelized network stack and the IIS 7.0. web server is multi-threaded with one thread per connection. II. A SIMPLE PERFORMANCE MODEL We can use use a well-known queuing theory result to compute the average response time of a request at a web server []. Assuming that requests arrive at the web server from a Poisson process, that a request s processing time at a server has a general distribution, and that a perfect loadbalancer equally distributes the load among all cores in the CPU we can use the M/G/ queue (that is a queue with Poisson arrivals, arbitrarily distributed service times, and a single server) to compute the average response time. Figure shows a web server with n identical CPU cores and a load-balancer that equally distributes the total incoming traffic of λ requests per second among all cores. The n cores in the CPU each have requests per second of processing capacity. The web server's total capacity is therefore n. CPU utilization is computed as: = λ E[ts] () Average queuing time Tq is computed as a sum of average wait time Tw and average service time Ts: Operating system Virtual machine (JVM or.net) Tq = Tw + Ts () For M/G/ system average wait time Tw is computed as: Application server Application Ts( Cs ) Tw = ( ) (3) Using () and (3) we can compute average queuing time as: Figure. Web server stack of layers Tq = Ts + Ts( Cs ) ( ) (4)

2 λ/n λ/n Equation can be used for computing of theoretical maximum throughput for complete CPU composed of n cores with processing capacity : λmax = n () If we insert Equations 6 and 7 in Equation 4 we have: λ LOAD BALANCER λ/n n Tq = + ( Cs ) n ( ) n (3) Figure. Multi-core web server architecture design includes n CPU cores, each with requests/sec of capacity λ - total traffic [request /s] - processing capacity for one core [request/s] n - number of CPU cores Cs is the coefficient of variation of the service time (the ratio between the service time s standard deviation Ts and the average service time Ts) Cs = Ts Ts If processing capacity for one core is marked as, then we can compute average service time as: Ts = (6) Arrival rate of requests for one core is: λw = n (5) (7) Little s law [4] let us compute the average number of requests processed as: Lq = λ Tq (4) We can interpret Equation 3 in many ways to analyze influence of some system's parameters on system's response time. As an example we can analyze how the number of CPU cores or Cs (coefficient of variation of the service time) can influence on CPU's response time. For an example we can analyze system with two CPU cores (n=), Cs = 5 (high value that is specific for web server) and processing capacity for one core that is 0 requests/sec. Plugging that values into Equation 3 for different values of total arrival traffic λ is resulting with different values of system's response time like on Figure 3. Figure 4 shows the variation in average response times as a function of the utilization for the same system from Figure 3. Canonical performance characteristics is occurring in all benchmark measurements (Figure 6) and it is placed under theoretical throughput characteristic with ceiling that is controlled by the bottleneck resource in the system and can be computed from Equation. Basic condition for stable service is: < (8) CPU utilization can be also computed as: = λw Ts (9) If we use Equation 6 and 7, we can compute Equation 9 as: = (0) n Figure 3. Canonical delay characteristic ( System's response time as a function of incoming requests ) Plugging the last Equation 0 in condition 8 yields the new condition for stable service as: λ < n ()

3 the Webserver Stress Tool ver. 7.0 benchmark to be our reference web workload [0]. Webserver Stress Tool is a benchmark that emulates large numbers of independent web clients. For every request web server is executing C# program code described on Figure 7. In addition to a web server, to isolate scaling bottlenecks specific to network processing, we conducted experiments using a workload with a trivial and computationally intensive web application just to use a maximum of CPU power and remove disk I/O traffic from the system bus and memory. Figure 4. Average response time as a function of the utilization protected void Page_Load(object sender, EventArgs e) for (int i = 0; i < n; i++) string originalstring = RandomString(stringlenght) string cryptedstring = Encrypt(originalString) string decryptedstring = Decrypt(cryptedString) Figure 5. Non-canonical throughput-delay curve (System's response time as a function of system's throughput) Figure 7. Test web page code description This web page will be opened every time for any request and in that case coefficient of variation of the service time Cs is equal. We have implemented three methods in class Page_Load : private string RandomString(int stringlenght) generates random string, length is defined with variable stringlenght public static string Encrypt(string originalstring) Figure 6. Canonical throughput characteristic On Figures 3,4 and 5 we can notice that average response time is getting smaller if we increase number of cores for the same incoming traffic. On Figure 6 we can see the point when system's throughput approaches n requests/sec (its maximum possible value) and system is getting unstable (number of waiting requests is exponentially increasing). In practice, as an optimal value for system's response time (from Figure 4), we will usually choose any value where system's utilization is between 0.6 and 0.8. III. EPERIMENT METHODOLOGY To determine how well a typical web server workload scales on a multi-core system, we chose a web server as our workload on an two and four-core system. We chose encrypts generated random string using DES algorithm public static string Decrypt(string cryptedstring) decrypts encrypted string In our test we have set up : n =0, stringlenght = We have tested two multi-core configurations from Table as web servers. Client configuration is described in Table. The testbed (Figure 8) was set up with client machine from Table and servers (systems under test) from Table, connected with Gbit full duplex 000BASE-T Ethernet connection.

4 Table. Server configurations SERVER CONFIGURATION Web server Web server CPU E80 NUMBER OF CORES 4 MEMORY 4 GB 4 GB Gbit Gbit NIC INTEL Pro INTEL Pro APPLICATION SERVER OS CPU Microsoft IIS 7.0 Windows Server 008 Table. Client configuration NUMBER OF CORES MEMORY NIC OS Microsoft IIS 7.0 Windows Server 008 Duo T7300 GB Gbit INTEL Pro Windows Vista Business In Equation 5 n is the number of cores and is processing capacity for one core (directly proportional to CPU s frequency). We can see that maximum throughput for CPU is,67 time bigger then for E80. That is within the limit because has 30% bigger working frequency and four cores (E80 has two cores). We can construct canonical delay characteristics for both systems on Figure 0. We can see that average queuing time Tq is reducing like in analytical model if we increase number of cores. With Equation 4 and Cs= we can calculate expected values for Tq using measured values for utilization and maximum throughput in Table 3. In Figure and Figure we have compared canonical delay characteristics from analytical model with measured, for both systems. We can notice from Figure and that analytical model that we used from Figure is good as a model of multi-core web server with two and four CPU cores with an assumption that income traffic is a Poisson process. Differences between analytical and measured results in Figure and Figure can be explained as a result of imperfect measurement of system utilization (We have used Microsoft Reliability and Performance Monitor). We have also used simple performance model and ignored amount of CPU's L cache. On real system (testbed configuration) we can't expect perfect load balancer assumed in model. Table 3. Test results for server configurations from Table 000 Mbit/s full duplex 000BASE-T connection Throughput [request / min] Average queueing time [ms] - Tq CPU utilization [%] Client with Web Server Test Tool Web server Figure 8. Testbed configuration We have simulated Poisson distribution in incoming traffic from client machine using Web Server Test Tool. Microsoft Reliability and Performance Monitor is used to measure CPU utilization. IV. TEST RESULTS Our test results are presented in Table 3. From Table 3 we can construct canonical throughput characteristics on Figure 9 for both servers. It is visible that maximum throughput for both servers is : λ E80 E80 E λmax = n (5) It is possible to calculate Ts from Equation 6 using the value of derived from Equation 5.

5 Figure 9. Measured canonical throughput characteristic Figure. Comparison of measured canonical delay characteristic for analytical model and system with four cores V. CONCLUSION Figure 0. Measured canonical delay characteristics Our results have shown that, due to flow-level parallelism in web server workloads, the number of cores is increasing performances of web server with an assumption that we have tested computationally intensive web application. We can notice that analytical model that we used is a good model of multi-core web server with two and four CPU cores with an assumption that incoming traffic is a Poisson process. We want to improve used model in future work to relate more realistic workloads for commercial web servers. REFERENCES Figure. Comparison of measured canonical delay characteristic and analytical model for system with two cores [] D.A. Menascé and V.A.F. Almeida, Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning, Prentice Hall, Upper Saddle River, N.J., 000. [] D.A. Menascé and V.A.F. Almeida, Capacity Planning for Web Services: Metrics, Models, and Methods, Prentice Hall, Upper Saddle River, N.J., 00. [3] L. Kleinrock, Queueing Systems: Volume I: Theory, John Wiley & Sons, New York, 975. [4] J.C. Little, A Proof of the Queuing Formula L = λw, Operations Res., vol. 9, no. 3, 96, pp [5] M. Andreolini, M. Colajanni, and R. Morselli, Performance Study of Dispatching Algorithms in Multi-tier Web Architectures, ACM Sigmetrics Performance Evaluation Rev.,vol. 30, no., Sep. 00. [6] V. Cardellini, M. Colajanni, and P.S. Yu, Dynamic Load Balancing on Web Server Systems, IEEE Internet Computing, May/June 999, pp [7] Frane Urem and Želimir Mikulić, Concurrency analysis of shared-memory multiprocessors, Proceedings of MIPRO 008 International Convention on MEET Conference, p. 4-7, 008. [8] M. Crovella and A. Bestravos, Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes, Proc. 996 ACM [9] Webserver Stress Tool ver. 7.0, Paessler AG, available at: