System Performance Improvement By Server Virtualization

Transcription

1 Sysem Performance Improvemen By Server Virualizaion Hioshi Ueno, Tomohide Hasegawa, and Keiichi Yoshihama Absrac Wih he advance of semiconducor echnology, microprocessors become highly inegraed and herefore muli-processor s are widely used. On he oher hand, only limied applicaion programs can use such muli-processors efficienly. In his paper, we show a mehod o improve uilizaion of muli-processor s based on virualizaion echnology, including he measured resuls of improvemens in example sysems. In some examples i was possible o muliply sysem of a. Index Terms concurrency, mulicore, muliprocessor, virualizaion, S I. INTRODUCTION ervers based on he x86 archiecure are now widely used, no only for simple s bu also for mission criical sysems in enerprises or governmen, because microprocessor has increased significanly. They are called IA-, Inel Archiecure, and ofen Microsof Windows OS or open source Linux OS is used on hem. Every year, microprocessor chips become more inegraed due o advances in semiconducor echnology. This resuls no only in increased single processor bu also in increased of shared memory ype muliple processor sysems, SMP: Symmeric Muliple Processor, where muliple CPU cores are inegraed on a processor chip. One facor of he direcion of echnology is he hea problem, high frequency clocks cause increased hea on he microprocessor chip. For his reason chip vendors don' develop higher single processors, bu muliply he CPU cores on a chip. Meanwhile, no all applicaion programs have efficien characerisics o uilize a single OS environmen on muliple processors. There are many informaion sysems which don benefi from muliple processor environmens. Sofware resource conenions preven concurrency of processes, if applicaion programs creae many processes or hreads. A virualizaion feaure may allow improved concurrency for applicaions wih limied scalabiliy by execuing muliple OS environmens on a muli-processor. However, i is known ha virualizaion feaures have some overhead. I is herefore recommended o Manuscrip received February 17, 2011; revised March 28, H. Ueno is wih Enerprise Server Division, Hiachi, Ld., Hadano, Kanagawa, Japan ( hioshi.ueno.uv@hiachi.com). T. Hasegawa is wih Enerprise Server Division, Hiachi, Ld., Hadano, Kanagawa, Japan ( omohide.hasegawa.v@hiachi.com). K. Yoshihama is wih Informaion Technology Division, Hiachi, Ld., Tokyo, Japan ( yoshihama_k@ig.hiachi.co.jp). evaluae he effec of concurrency wih using virualizaion by sudying is advanages and disadvanages. In his paper, we discuss abou improvemens of sysem by dividing s which have a large number of physical CPU cores, ino a number of LPARs which have a smaller number of CPU cores, in our example using he virualizaion feaure "Virage" implemened on Hiachi BladeSymphony s. We show ha i is possible o improve sysem by using virualizaion even for ough applicaion programs wih limied concurrency in muli-processor environmens. II. EFFICIENT MULTI-PROCESSOR SERVER USE CASE USING SERVER VIRTUALIZATION FEATURE Tradiional use of IA sysems in a business environmen uses a simple design approach where one subsysem is buil using one OS environmen of one physical. For example, if you wan o build wo subsysems, like a CRM subsysem and a Sales Managemen subsysem, you deploy wo s, one wih he CRM applicaion and anoher wih he Sales Managemen applicaion. However here are many inefficien applicaions for muli-processor s because no all CPU cores on he s can be used. In ha siuaion, a srange phenomenon can be observed, where mos resources such as CPU, sorage, nework, are idle, bu sysem hroughpu can be increased. In his paper, we call physical processor chip as "processor" and we call each insrucion execuion uni in he physical behavior of sofware four processes waiing processes [b][c][d] four physical CPU cores #0 #1 #2 #3 a b c d ime execuion ime idle ime CPU core usage coun = 5/4 = 1.25 CPU core busy raio = 5/(4*4) = 0.31 Fig. 1. An example of muliprocessing behavior. Sar of he ransacion End of he ransacion

2 processor as "core" or "CPU core." I means ha here are muliple cores in one processor. Here we hink abou characerisic applicaion programs which have only a single process concurrency, even if he program runs on a four core muli-processor. If no oher resources excep he processors are bole neck for he, 100% busy of one core means reaching he upper limiaion of. In his case, he processor usage raio is, 1 (core)/ 4(core) = In general, we assume ha N is number of cores and c(p) is he average number of cores which he applicaion programs use, he processor usage raio is c(p)/n. Values of c(p) are differen and depend on characerisics of he OS and/or he applicaion programs. The maximum value is N for high concurrency programs, and lowes value is 1 for low concurrency ones. I is no easy o develop applicaion programs which can use muli-processor s efficienly[1][2]. Mos applicaion programs use muli-processing or muli-hreading for beer efficiency, bu i is rare for processes o be able o run compleely independen each oher. The reason of he phenomenon is exclusive conrol like lock mechanisms for common resources in he applicaion program. Dependency beween processes can also he reason of i, where a process has o wai o receive from anoher processes for synchronizaion. Fig. 1. shows an example, a ransacion processing sysem which has four processes. The process "a" receives ransacions from oher sysems, i makes some compuaions and sends he processed o wo oher processes, "b" and "c" for oher processing. Afer ha he process "d" wais for receiving from processes, "b" and "c". In such a siuaion, he sysem has physically four cores, bu average processing abiliy is 1.25 cores, because each process consumes seconds wih using one core, and his ransacion oally consumes 5 seconds bu elapsed ime is jus 4 seconds. In his case, core use raio in whole is 0.31 ha is 1.25/4. In case of low concurrency applicaion programs, in oher words very small c(p)/n case, usually he sysems manager wans o use high speed single processor o keep good sysems, he does no wan o use relaively slow muli-processor s. As we described before, he developmen of high single processor s is difficul, and he availabiliy of such s is limied. So as a maer of fac many users have o achieve he required using muli-processor s. You have o divide he arge sysem as well as he ino muliple small sysem insances, and hen run hese sysems in parallel, if you have o achieve he required using s wih a slower clock. Each divided sysem works independenly, and i becomes a kind of load balancing sysem configuraion. However i is difficul for some applicaions o run muliple insances on a single OS of he same, because here are many programs which can run muliple insances on one OS, as hey may collide o ge he same resources from he OS. Generally i is beer o use virualizaion echnology for muliple OS environmen on a, because differen physical s for applicaion programs increase low CPU core usage rae s which is no efficien. I is known ha virualizaion echnology has some loss of processor by conrol overhead like emulaions. Accordingly he oal improvemen raio depends on he virualizaion overhead and he efficiency of program concurrency, on whole by dividing OS insances. In he following secion, we describe an evaluaion mehod for finding relaions of one sysem's and oal of muliple s. III. EVALUATION OF MULTIPLE LOGICAL SERVER The comparison is made by comparing he of one OS insance on a and oal of n OS insances on n s. Here, we define sysem as "Sph," and as "Sv(1) Sv(n)," and hen we undersand relaions among and as follows, and assume ha he number of cores assigned o s are sufficien for he evaluaion of arge applicaion concurrency. (Fig. 2.) (a) The difference of on one case, Sph Sv 1 is simply affeced by he virualizaion overhead. The difference beween of i s and ideal which is i imes of he one, i Sv 1 Sv i, is he overhead agains 's scalabiliy. (c) Define iovc as he number ha saisfy he condiion of s ha oal CPU core coun exceeds physical core coun, and ha condiion is called "CPU over commimen" condiion. On his condiion area, we can consider ha he difference beween he ideal scalabiliy and he of iovc s is from physical CPU core access overhead caused by sharing core for cores of LPARs. We define improvemen raio as max 1,. If i is less han 1, i means no improvemen. However, even in he bes case, i never exceeds he number of physical cores N, his bes case being an applicaion ha does no have idle ime. When he applicaion has no concurrency and can use one core only, we can divide he oal sysem Sv(3) Sv(2) Sph Sv(1) (a) ideal scalabiliy number of s (a) virualizaion overhead scalabiliy overhead by hypervisor conrol (c) limiaion by CPU over-commimen Fig. 2. Model of characerisic. (c)

3 64 CPU cores load program wih wo hreads (a) configuraion of physical evaluaion hypervisor configuraion of virualizaion scalabiliy evaluaion The has eigh Inel X7560 processors and 64 CPU cores (The is configured by four blades inegraed ino one SMP. I is implemened in Hiachi BladeSymphony BS2000 and "Virage pariioning hypervisor) Each is assigned wo cores by he hypervisor. Fig. 3. Configuraions of he simple workload case. process of he applicaion program ino N processes, and he is N imes ha of a wih one core. IV. EXAMPLES OF SYSTEMS EVALUATION In his chaper we sudy hree cases for improvemen raio evaluaion. A. Simple Load Case A firs we sudy simple applicaion program behavior, in his case only CPUs are used by he program, I/Os are no, and CPUs execue only user mode insrucions, bu don' execue supervisor mode insrucions, as hese migh generae virualizaion overhead. If an applicaion program uses only C cores on a, he maximum raio o M s which has C cores is expeced o be C: M C. In his secion, we show he measuremen resul of a simple applicaion program o verify above heory. The simple applicaion program runs wo cores (C=2) wih using wo hreads, and i uses user mode insrucions only o exclude he effecs of hypervisor inervenion. From one o oal sysem (only 2 cores used) (2 cores for each LPAR) remoe file 1.9GB 1.9GB 1.9GB 1.9GB 1.9GB 1.9GB sorage 1.9GB Virus scan sofware (a) configuraion of physical evaluaion Fig. 5. Configuraion of virus scan sysem case. hypervisor configuraion of virualizaion scalabiliy evaluaion The has wo Inel X5460 processors, 8 CPU cores. (Hiachi BladeSymphony BS1000 wih Virage hypervisor) Each is assigned one or wo cores by he hypervisor. The hypervisor suppors up o 16 s on a. 40 s M 1 40 are used for his experimen. (Fig. 3.) Fig. 4. shows he resul of he experimens. This program execues a cerain insrucion rouine many imes, and measures by couning number of loops per a second. Because no even an OS is no used for his program, lile virualizaion overhead is moniored herefore physical single is abou he same as single. We clearly see he scalabiliy of i, because he oal of 32 s is 31.7 imes ha of a. The oal is in proporion o increases of he number of he s M. The reaches a ceiling above M=33, because of he CPU over commimen condiion. This has 64 cores and in he M=32 case, all s wih wo cores each use he oal of 64 cores. B. Virus Scan Sysem Case A pracical example of sysems improvemen using virualizaion is a virus scan sysem. This is a nework file sysem and virualized virus proec s raio number of s Physical has 64 cores. For up o 32 LPARs ideal scalabiliy can be seen, because physical resources are sufficien. Over 32 LPARs he scalabiliy is limied. N of sv. (M) Nof CPU cores (C) oal perf. * for each for each for each for each for each for each for each for each *1 loop coun (*10 4 /sec) Fig. 4. Measured scalabiliy for he simple workload case. 5.0 (a) number of s Virualizaion overhead(a) exiss for single case, bu oal hroughpu is exremely high in case of 6 s. Scalabiliy overhead is relaively small. N of sv. N of CPU cores exec ime file (GB) perf raio phy. sv for each log. sv. 3 1 for each for each (4 sv), 2(1 sv) (4 sv), 2(2 sv) exec ime is he average of all ime. perf. raio = file size(mb)/exec ime(sec) Fig. 6. Measured scalabiliy on he virus scan sysem case.

4 scanning he whole sorage. I is a kind of nework file sysems, and here we sudy he scanning hroughpu of whole sorage by virualized virus proecion s. In a file sharing sysem of an enerprise, he sysem managers check all files once a week o mainain securiy, bu his is no easy because he oal file size is very large and i akes much ime o complee scanning. Scanning ime has o be as shor as possible, because he check has o be compleed in limied ime during he weekend. When performing a virus scan wih a, is processors are no so busy because hey wai for sorage access ime or resources exclusive conrol by he applicaion program. Muliple virus proecion programs can' be acive on a, so we can' increase he processor usage rae. For his sysem, we can shoren scan execuion ime by using concurren virus proecion program on a divided ino muliple s and OSs by virualizaion. In his evaluaion case, we made a configuraion which has a, 1.9GB shared disk (Fig. 5.(a)), and we made anoher configuraion which has maximum six s on a, six 1.9GB shared disks for each for scan arge (Fig. 5.), and we measured heir ime o scan. Fig. 6. shows resuls of he measuremens. In one case, scan by a is 47% beer han i by a. However once wo s are used he is 36% more han for he physical, for six s, he is 3.9 imes higher. From a view poin of he scalabiliy, i is good for six s o have 5.7 imes he of a single. In his case, number of s is six, oal CPU cores are eigh, herefore we see i does no reach o he upper limi of CPU over commimen condiion. C. Mixed Applicaion Program Case In his secion, we show resuls of measuremens when four kinds of applicaions are run on a virualized ino four s. We chose web, DB, Java applicaions and mail sysems as four kinds of applicaion programs because hese are he applicaions used widely around enerprises. These sysems are implemened each on a separae s. Obviously, we can compare he of differen applicaions wih each oher, bu we have o evaluae oal scalabiliy when we increase he number of applicaion program for workload 1 core for each workload se web 8 cores DB Java mail hypervisor configuraion of virualizaion scalabiliy evaluaion The has wo Inel X5570 processors and 8 CPU cores. (Hiachi BladeSymphony BS2000 and "Virage hypervisor) Each is assigned one core in shared mode. Fig. 7. Configuraion of he mixed sysems case. raio (a) number of s Virualizaion overhead(a) exiss in 4 s case. Scalabiliy overhead is small on 8 s case. Limiaion by CPU over commimen (c) (Sum of CPU cores exceeds physical cores.) four s workload se N of core Geo. sv. *1 web Java DB mail mean phy. sv log sv *1 N of cores for each physical/ Values are normalized by. One workload se is configured by four s. Toal is calculaed by geomerical mean. Fig. 8. Measured scalabiliy on he mixed sysems case. s. Therefore, four s corresponding four sysems are defined as a workload se, and four, eigh and welve s cases are being evaluaed (Fig. 7.). Fig. 8. shows he resuls. Those values are normalized by for each applicaion. In case of four s, he oal is 15% lower han for s, bu i is 60% higher for eigh s and 84% higher for welve s. In he case of welve s here is CPU over commimen condiion which uses oal 12 cores on 8 physical cores. Physical resources are no sufficien, bu he is sill increasing because each sysem does no consume cores a 100%, due o idle ime. V. ANALYSIS OF PERFORMANCE IMPROVEMENT A. Performance Scalabiliy of Virualizaion Here we analyze he improvemen raio of he hree sysems described in he previous chaper. Table I shows and he condiions under which hose sysems are no in a CPU over commimen sae. Column (a) shows he raio of physical and s for a single or single workload se environmen. Alhough i depends on kinds of applicaion or processor ypes used in he, we see virualizaion overheads beween 0% and a maximum of around 30%. The value in column (c) shows he increase for muliple s or workload ses agains a. In all cases, he achieved by a (c)

5 case TABLE I PERFORMANCE ANALYSIS OF THREE EXAMPLE SYSTEMS iem 1 workload se workload se a maximum core V:P perf. raio N of cores oal perf. raio for phy. sv. forvir. sv. N of cores N of workload se scalabiliy in virualizaion (a) (c) (d) (e) (f ) (g) Simple Virus scan applicaions N of workload se : he number of LPARs or workload ses number of s is beer han a single physical which indicaes a improvemen by virualizaion. Column (g) indicaes he scalabiliy, how sysems increases in proporion o he number of s or workload ses (f) when some s are added. I is calculaed by column(d) / column(f). Under opimal circumsances, he value is 1. Inuiively i means "he is close o n when we prepare n unis of s" and we define i as he scalabiliy indicaor in a virualizaion environmen. In all cases, i is over 0.95, herefore we undersand he scalabiliy is almos opimal. Wih he above sudy, we verified ha, in an environmen wih sufficien physical resources, muliple s can achieve increased sysem compared wih a running a single OS. We undersand also we are able o saisfy required by preparing enough number of s for our sysems. B. Evaluaion Mehod for General Use Case Generally, we can esimae he improvemen raio by following calculaion seps ha are based on characerisics of he as described before. sep 1. Run he arge sysems and measure Sph as he of applicaion programs on a physical. Measure processor usage raio "P" on he also wih monior funcions of he OS. Here we calculae he subsanial number "C" of cores. We assume "N" as he number of physical cores. C N P sep 2. Run he arge sysems on a by using he same OS and sofware, and measure he of applicaion programs as Sv(1). Here he mus be assigned more han C cores calculaed above. The number of cores assigned here, we assume "Cv". sep 3. Calculae he maximum sysem a CPU non over commimen condiion area, because in he area virualizaion scalabiliy is good as we discussed before. The maximum number of s under his condiion is calculaed as Lv N/Cv and ha he maximum can be esimaed as Esimaion of Sv Lv Sv 1 Lv R. Here "R" is he coefficien deermined by kinds of applicaion programs and i indicaes he characerisics of scalabiliy, as in column (g) of TABLE I. Afer he calculaions as above seps, if he number of Lv is larger han 1, and esimaed sysem Sv(Lv) is large enough compare wih Sph, we can assume ha he virualized sysem will show improvemen from he. In pracical business sysems, finding P is relaively easy, and as a resul of ha deermining Lv is also easy. Measuring Sv(1) is no so easy when using a sofware based virualizaion environmen, bu i is relaively easy in case of a pariioning ype virualizaion feaure like Virage. You can make survey he virualized afer simply change he mode from physical o virual, because he pariion's disk forma is same wih 's forma. The R value is expeced o be in he range from 0.9 o 0.99 as we measured a hree example sysems, bu if your applicaion program's R is unknown you may esimae i smaller. Confirmaion of improvemen using virualizaion is possible by he calculaions we showed here. VI. CONCLUSION In his paper, we have described ha one 's will be increased by improving concurrency using virualizaion echnology, because here are a many applicaions which can use muli-processors efficienly. Essenially, i means ha virualizaion assiss improvemen of efficien resource uilizaion, bu we can also say ha he virualizaion echnology can improve 's. How he will improve depends on characerisics of applicaion programs. Virualizaion overhead may be sriking, if he arge programs already have high concurrency on a. When you apply virualizaion o your sysems, you should evaluae heir characerisics carefully, bu his mehod o improve muli-processor's is very imporan in fuure, because microprocessor echnologies advance in he direcion of muli-processor and muli-core. We will coninue o sudy mehods of evaluaions as well as mehods o improve sysems for virualizaion environmens. REFERENCES [1] Angela C. Sodan, e.al., Parallelism via Mulihreaded and Mulicore CPUs, IEEE Compuer, Vol.43,No.3: pp (2010). [2] Wu-chun Feng, e.al. Tools and Environmens for Mulicore and Many-Core Archiecures, IEEE Compuer, Vol.42,No.12: pp (2009). [3] H. Ueno, e al.; Virage: Hiachi s Virualizaion Technology. In: GPC 2009 Workshop Proceedings of Conference, pp IEEE-CS, Los Alamios (2009) [4] H. Ueno, e al.; Virage: Server Virualizaion wih Hardware Transparency. In: Euro-Par 2009 Workshops, LNCS 6403, pp , Springer-Verlag Berlin Heidelberg (2010) [5] P. Apparao, e al.: Archiecural Characerizaion of VM Scaling on an SMP Machine: ISPA 2006 Workshops, LNCS 4331, pp , Springer, Heidelberg (2006) [6] H. Umeno, e al.: Developmen of a High Performance Virual Machine Sysem and Performance Measuremens for i: IPSJ J. Informaion Processing, Vol.4 pp (1981) [7] J.E. Smih, R. Nair: The Archiecure of Virual Machines: IEEE Compuer, pp (2005) [8] R. Uhlig, e al.: Inel Virualizaion Technology: IEEE Compuer, pp (2005)

6 [9] J.E. Smih, R. Nair: Virual Machines, Versaile Plaforms for Sysems and Processes, Elsevier Inc. (2005) [10] A. Barham, e al.: Xen and he Ar of Virualizaion: Proc. The 19h ACM Symposium on Operaing Sysems Principles, pp (2003) [11] H. Umeno and S. Tanaka: New Mehods for Realizing Plural Near-Naive Performance Virual Machines: IEEE Transacions on Compuers, Vol. C-36, No.9, pp (1987) [12] N. Gil, e al.: Inel Virualizaion Technology: Hardware suppor for efficien processor virualizaion: Inel Virualizaion Technology, Vol 10, Issue 03 (2006)