The Advantages Of A Nformatonal Nair System

Transcription

1 Effectve Use of Resources Dstrbuted Cloud Computng Platform for Provdng Qualty Multmeda Servces Irna Bolodurna Dens Parfenov Orenburg State Unversty , Pobedy avenue, 13,Orenburg, Russa Abstract. Exstng approaches to the use of cloud computng resources s not effcent. Modern multmeda servces requre sgnfcant computng power, whch are not always avalable. In ths paper, we ntroduce an approach that allows more effcent use of lmted resources by dynamcally schedulng the dstrbuton of data flows at several levels: between the physcal computng nodes, vrtual machnes, and multmeda applcatons. Keywords: cloud computng, cloud system, computng node, computng resource, hghload nformaton systems, load balancng, qualty of multmeda servces, vrtual machne, vrtual resource component. 1. Introducton The nformaton flows between computng nodes n local and global networks has been steadly ncreasng each year. It s true not only for large data processng centers, but also for locally datacenters (DC) specalzng n ndustry, economy, health and so on. An mportant area to use local DCs s educaton. Unverstes are ncreasngly usng ther own DCs to support ntegrated automated nformaton systems (IAIS), provdng end users wth network multmeda servces. The need for more resources s one of the problems of hgh-loaded IAIS. The consumpton of resources unlke the avalable volumes grows exponentally. [5]. The analyss of request flows to IAIS servces shows ther structure heterogenety [1]. Modern IAIS servces are based on the concept of cloud computng. However, the problem of lmted resources used for cloud systems remans relevant [4]. The use of vrtualzaton and cloud computng allows to consoldate several onlne servces located on vrtual machnes (VM). It reduces the number of physcal servers. But to effectvely deploy applcatons on VM t s necessary to solve the problem of resource plannng based on varable loads and servce level agreement (SLA) [3]. The most flexble archtecture of cloud computng s the nfrastructure as a servce (IaaS). Ths archtecture allows the user to control a pool of computng 315 Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, resources. Ths approach can mply the start of operatng systems and applcatons, and the creaton of vrtual machnes and networks. Thus, cloud computng leads to sgnfcant cost savngs due to the ncreased load densty [2]. However, the above s not enough to consoldate computng power, to reduce the nfrastructure overheads and to reach optmal performance of cloud systems. To use the cloud nfrastructure effectvely new methods and algorthms should be developed to control components of cloud systems. It demands determnng the formal structure of a cloud system [6]. 2. Model of resource vrtualzaton of cloud systems In our research, we have developed a model of computng resources of cloud systems. The concepton of vrtualzaton of computng resources s based on abstractons representng the tuples of relatons between the nterconnected elements of subsets. The cloud system can be represented as a set of nterconnected obects. They are computng nodes (Snode), system storages (Sstg), network attached storages (Snas) and schedulng servers (Srasp). The number of obects and the content of each set may vary dependng on the cloud s sze and ts use. Each compute node can run multple nstances of vrtual machnes represented as a set: Snode ={VM,1, VM,2,, VM,k }, (1) where k s the number of vrtual machnes on a compute node, = 1...l (l number of nodes). Each vrtual machne belongng to the set (1) can support several applcatons and servces represented as a set: VM ={App,1, App,2,, App,n }, (2) where n s the total number of applcatons and servces, =1... m (m - number of VMs). The network attached storage ncludes a set of predefned VM mages. Snas y ={VMmg y,1, VMmg y,2, VMmg y,p }, (3) where y = 1... z (z - number of network attached storages). Each VM mage contans an operatng system wth prenstalled software and predetermned hardware parameters. VMmg y,z ={OS 1, OS 2, OS r }, (4) The work of entre cloud system s performed usng the plannng system for certan operatons defned by the schedulng servers. Srasp={Rtask 1, Rtask 2, Rtask f }, (5) The dstrbuted storage system usually conssts of falover RAID arrays Sstg f ={RDsk 1, RDsk 2,, RDsk d } contanng the nformaton for multmeda servces 316

2 RDsk d ={Data 1, Data 2,, Data s }, (6) In addton, the cloud system also contans vrtual and physcal swtches for nterconnecton between all the components n a network. Each component of a cloud system Shcn={Snode, Snas, Srasp, Sstg, VM } has the followng characterstcs: Shcn=(State, Mem, Dsk, Dskn, Core, Lan), (7) where State { on, off } s the state of the component; Mem N s the sze of RAM; Dsk N s the dsk capacty for storage; Dskn N s the number of storage devces; Core N s the number of processor cores; Lan N s maxmum bandwdth of the network adapter; The set of vrtual machnes can be dvded nto subsets VMnode={Snode, Snas, Sstg, } to solate computng resources for dfferent servces from each other. The cloud system s a dynamc obect changng at tme t. Its state can be formalzed n an orented graph form: Shcn(t) = (Node (t), Connect(t), App(t)), (8) where Node(t)={Node 1,Node 2,,Node } are actve elements ncluded n one of the sets Snode, Sstg, Snas k, Srasp m ; Connect(t)={ Connect 1, Connect 2,, Connect } are actve connectons by users to the vrtualzed applcatons; App(t)={App 1, App 2, App n } are actve nstances of applcatons runnng on vrtual resources. So we determne the structure of a cloud system and mechansms of ts component nteracton. In such a system smultaneous servcng heterogeneous user requests s not trval task. To optmze the mechansm of access to nformaton system resources t s necessary to analyze the man data flows transferred wthn the cloud system. Model of data flows n hghload nformaton systems based on cloud computng For flows analyss n our study, we used nformaton systems of educatonal nsttutons. For analyss the most popular multmeda servces have been determned. The research consdered dstance educaton systems (DES) consstng of dfferent nteractve applcatons. In our research has bult a level classfcaton of applcatons: Level 1: The subsystem for montorng the students' knowledge n real tme; Level 2: The subsystem of the electronc lbrary; Level 3: The subsystem of webcasts and webnars. 317 Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, In our study, we have determned the general features of the use of the local DC s equpment. the load on the key resources s perodc and rregular; requests to multple types of resources come at the same tme; load dstrbuton s not optmal, whch results n loss of servce at peak loads; up to 90% of the load s predetermned, as pre-regstraton s used for access to resources; up to 70% of the load arses due to multmeda educatonal resources. Informaton flows at each level have ther own characterstcs. The ntensty of servcng requested flows n the nformaton system depends on the target applcaton level. In a study we use the statstcal analyss of the load on the most popular applcatons used n nformaton systems of the unversty. Evaluaton tme for requests to varous applcatons allow to forecast flows and ensure effcent allocaton of resources. We usng the goodness of ft ch-square Pearson to obtan data to test the hypothess of dstrbuton laws requests for ncomng flow. In general, the ntensty of ncomng and servce of a request flow for each class of applcatons s determned by the dstrbuton functon, whch s descrbed by the followng dstrbuton laws: for level 1 - Ch-squared dstrbuton; for level 2 - Webull dstrbuton; for level 3 - Pareto dstrbuton. Flows of data transmtted n the IAIS are usually processed n several phases. At the same tme n each phase several smlar elements can be used provdng balancng and load sharng between the components of the nformaton system. The number of components n each phase depends on the functonalty of the nformaton system and the number of applcatons ncluded n ts composton. Suppose an nformaton system has the form: IS { S 1,, S r } (9) where S - a component that performs data processng on the bass of the ncomng flow of user requests, = 1..r (r the total number of components of the nformaton system). The number of phases f n the flow path of user requests n an nformaton system depends on ts archtecture. The purpose of each phase accordng to ts locaton n the processng sequence s: The frst phase s the dstrbuton of data flows between the IAIS resources n the cloud; The second phase s the dynamc scalng of the computng resources n the cloud; The thrd phase s data processng by user applcatons usng storage systems and databases. 318

3 The components of the thrd phase nclude nodes of storage systems and database management systems for provdng access to multmeda servces n the cloud. In detal the set of components of an nformaton system s represented n form: IS={S 1 1,,S 1 n, S 2 1,,S 2 m,s 3 1,,S 3 k }, (10) where S s the component of the phase; mn, nn, kn are the numbers of components ncluded n the system for the respectve phases f. We also ntroduce the nput components S 0 whch transmt data flows nto an nformaton system, and output components S 4 recevng data flows from the cloud nfrastructure. Consequently, the set descrbng the nformaton system s transformed to: IS={S 0 1,,S 0 l,s 1 1,,S 1 n, S 2 1,,S 2 m,s 3 1,,S 3 k S 4 1,,S 4 p}, (11) where pn, ln are the numbers of components n the nput and output of cloud nformaton system. Each component S of the nformaton system at any tme can servce multple requests from dfferent users. In the process of the user request data flows are generated upstream and downstream of the component. Ther ndvdual characterstcs vary n tme. We desgnate all the ncomng flows of component S as X, and the outcomng as, where s the number of the components at the servce phase. Each request flow can be descrbed as a set of characterstcs. Suppose, there are l ncomng flows and p outcomng flows for a component S. Then for the ncomng flow =1.. l, we ntroduce a set of characterstcs: (, ) (, ) (, ) ( t) x1, ( t),, xk, T X ( t) (12) where (, ) x s the ntensty of recevng requests n each ncomng flow of the 1, component x (, ) 2, (, ) 3, S ; s the servce tme of the request flow v of the component S ; x s the ntensty of servcng requests of the request flow v of the component S ; (, n) x s the servce dscplne of the flow of 4, S, whch determnes the order of servce n accordance wth the prortzaton algorthm n the nformaton system; 319 Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, x s the servce class of the flow of (, ) 5, (, ) 6, 320 S ; x s the number of requests receved from the flow of For outcomng flow =1.. (, ) (, ) (, ) 1, k, p of the component ( t) y ( t),, y ( t) T S.. S the feature set ncludes: (13) The servce path for each flow can be dynamcally changed. The number of unque flows depends on the number of components n each phase. A set of ncomng flows at each phase can be represented as: X n 0 X (14) where s the number of the servce phases, n s the number of flows at phase. Consequently, all the ncomng flows of the nformaton system can be represented as: f 0 X X (15) where f s the number of servce phases. For output flows the smlar condtons are used : n f 0 0 (16) To effectvely serve user requests formng data flows n the nformaton system, there must be an sngle-valued mappng of the form R : X. In addton, for servce of any request at each moment of tme the matrx H of transtons between the phases of servce s constructed dependng on the class of the request and the current load of the system. The graph of transtons between phases can be bult usng the functon: 1, 1 e R( X ), e where e s the component of phase -1 drectng data flow to component phase, =1.. l. (17) S of Then for any component S the set of all the nput flows receved from component 1 S located n the prevous phase s represented n the form: X R R( X ) (18),

4 where s the phases of servce. Then effluents element, 1 1 * X n X * 0 S drected to the element 1 S represented n the form: R( X ) (19) So and m can descrbe the ncomng and outcomng flows 0 of phase respectvely. In real systems, outcomng flows can overlap and get servced on the same computng node that results n the formaton of nternal queues at each servce phase. To descrbe ths process t s necessary to determne the connectons between output flows of component S at phase and all the components at phase +1. * Consderng the above the set becomes: *,0 S S 1, 1 For a descrpton of ntersectng ncomng flows wthn one phase two functons are ntroduced:, 1 * x ( X Q, 1 * y ( Q * ) ) where Q x ( ) characterzes nput ntersectng flows and Q y ( ) characterzes output ntersectng flows for phase +1. Smlarly, a set of nput flows enterng the phase of servce can be defned. The flows of user requests can also ntersect. Consequently, an nput data flow arrvng on the component S at phase from all the components at phase -1 can be represented as: * (20) (21) (22) *,0, 1 X X X (23) S 1 S To descrbe the ntersectng flows from the phase we ntroduce two functons:, 1 * X P ( X ) (24), 1 * y ( X P x ) (25) Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, * where P x ( X ) characterzes ntersectng nput flows, and P y ( X ) characterzes ntersectng output flows from phase -1. Thus, the functons (21) and (25) descrbe the data flows between phases of servce n an nformaton system wthn a cloud. To descrbe the whole multphase nformaton system we formalze the descrpton of flows n each phase n the form R : X. Thus data flows n an nformaton system wthn a cloud can be represented as: R( X ), X X * *,0, 1 R ( X ) P ( X ), X X X (26) 1 S S * *,0, 1 Q X ( ), 1 S S Data flows and ther characterstcs may change over tme and our representaton thereof should also nclude tme t. The descrpton of an nformaton system should nclude both nternal and external factors so the parameter of external nfluence F should be ntroduced. Then data flows n a cloud system can be descrbed n the form: R ( X, t, F) (27) 3. Cloud system vrtual resources control algorthm The above models allow to determne the most approprate computng nodes of the nformaton system and the vrtual machnes that contan the requred nstances of multmeda applcatons. The control system should provde unnterrupted user servce and effectve vrtual resource control n case of lmted physcal resources. The man task of the control system s schedulng of computng resources at each moment of tme. For hghload nformaton systems effectve schedulng s mportant because the load on the servces may vary greatly wthn short tme ntervals. In a cloud system there s a need to plan resource consumpton optmally to prevent resource exhauston for the applcaton already runnng. As dstnct from other nformaton systems the flow of user requests n the educatonal envronment s predctable due to the subscrptons for multmeda servces. The control algorthm for user access to vrtual nformaton resources conssts of two nterconnected processes. One of these processes s schedulng. The schedulng algorthm collects data on the ncomng requests and classfes them by the levels determned wth the prortes of applcatons for busness processes. The nput data for the algorthm are the applcatons descrbed accordng to the template that ncludes a vrtual machne *

5 mage wth the gven confguraton of hardware and software and user sesson characterstcs. Based on ths template and data analyss of connectons the algorthm calculates the confguraton to deploy the requred servce. In the case of dentcal sets of VM software the already stored mages are used. To optmze the use of computng resources the algorthm generates three varants of vrtual machne confguratons. The frst varant provdes reserve performance n the case of unexpected ncrease n the number of users. The scalng factor n ths case s calculated dynamcally. The second varant provdes a predetermned low performance of vrtual machnes for the gven number of users. Ths approach s most effectve for small specal purpose user groups. It allows to reduce the overhead n case few workng users, the number of subscrbers beng large. The thrd varant uses user-predetermned characterstcs, ncludng a fxed number of runnng nstances of vrtual machnes regardless of the number of users. In ths case the algorthm s only used to lmt the computng resources. It calculates the maxmum number of vrtual machnes that are avalable n the confguraton selected by the user. The second process wthn the algorthm s drect servce of user requests and resource scalng durng the work of applcatons. The algorthm consders the total number of requests from each source whch allows to predct the load on the runnng applcatons wthn the cloud. Then the algorthm mgrates vrtual machnes between computng nodes based on the collected data n accordance wth a predetermned plan, thereby scalng the work of applcatons. For effcent use of resources wthn the above processes, addtonal nstances of vrtual machnes are created n the onlne storage of mages for support the applcatons provdng an access for the mnmum amount of users. In the case of predcted load ncrease on a certan servce, the algorthm deploys a full mage of the meda resource and analyzes the ncomng user requests. If the load does not exceed the number of queres n an ordnary flow, the algorthm swtches the load to the approprate mage and turns off the vrtual machne. The scheme of an ntegrated approach to optmzaton usng cloud computng, s presented n fgure 1. Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, Fg.1 Scheme of optmzng access to nformaton system based on cloud computng Our approach allows to consder the physcal lmtatons of computng resources and organze the work of a cloud nformaton system adustng the number of nstances of runnng applcatons based on the ncomng flow of user requests. 4. Expermental part We have studed the work of the cloud nformaton system wth dfferent parameters to evaluate the effectveness of our vrtual resource control algorthm. We have used the standard algorthms from the cloud system OpenStack [5] as reference for comparson n the experment. In the experment, we used the flow of requests smlar to the real flow wthn the nformaton system of dstance learnng. The number of concurrent requests receved by the system was about 10,000, whch s equal to the maxmum number of potental users of the system. All the user requests are classfed nto sx user groups correspondng to the types of user behavor. The requests from the frst three user groups drected to the allocated applcaton usng other applcatons at the same tme. The groups from 4 to 6 smulate the work of the applcaton n the case of computng resource shortage because of an excess number of concurrent requests. The ntensty of usng the system components (vdeo portal, testng system, and electronc lbrary) and the amount of the requested data were assgned for each user group. Experment lasted for one hour whch corresponds to the longest perod of peak load n the real system. Expermental results are presented n the Table

6 TABLE 1. Servce effcency of user requests Systems Testng electronc vdeo testng electronc vdeo system lbrary portal system lbrary portal Experment 1 3 Number of requests Volume of nformaton Number of servced requests (wthout load balancng) 5443 (4352) 622 (418) 517 (356) 592 (465) 643 (512) 4320 (3985) The ntensty of servce 90,71 (72,53) 10,36 (6,96) 8,61 (5,93) 9,8 (7,75) 10,71 (8,5) 72 (66,4) Experment Number of requests Volume of nformaton Number of servced requests (wthout load balancng) 632 (525) 5384 (4625) 560 (376) 6753 (5642) 6351 (5215) 5860 (4129) The ntensty of servce 10,5 (4,2) 89,73 (77,08) 9,3 (6,26) 112,5 (94,03) 105,85 (89,91) 97,6 (68,81) The results of the experments show a decrease of 12-15% of the number of servce denals n accessng to multmeda servces wth lmted resources. Wthn the experment n the OpenStack cloud system we compared the consumpton of vrtual resources by the number of vrtual servers for each of the subsystems. Our control algorthm provdes collaboratve work of all runnng nstances of applcatons n accordance wth user requrements due to the optmal allocaton of resources on each computng node. So the optmzaton algorthms may release 20 to 30% of the allocated resources (vrtual servers) (Fg. 2). Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, Besdes the reducton of the number of allocated vrtual resources allows to scale a cloud system more effcently and provdes a reserve for the case of ncrease n the ntensty of usng applcatons. Authors thank for support the Russan Foundaton for Basc Research (proect A). References [1]. Qnga Huang, Ka Shuang, Peng Xu, Jan L, Xu Lu, Sen Su Predcton-based Dynamc Resource Schedulng for Vrtualzed Cloud Systems Journal of Networks, Vol 9, No 2 (2014), , Feb [2]. S. J. E. C. I. C. Clark, K. Fraser and A. Warfeld, "Lve mgraton of vrtual machnes, " In Proc. NSDI, [3]. N. Bobroff, A. Kochut, and K. Beaty, "Dynamc placement of vrtual machnes for managng SLA volatons," n Integrated Network Management, IM'07. 10th IFIP/IEEE Internatonal Symposum on. IEEE, 2007, pp [4]. Q. Huang, S. Su, S. Xu, J. L, P. Xu, and K. Shuang, "Mgraton-based elastc consoldaton schedulng n cloud data center," n Proceedngs of IEEE ICDCSW [5]. A scalable nfrastructure for CMS data analyss based on OpenStack Cloud and Gluster fle system S Toor et al 2014 J. Phys.: Conf. Ser [6]. Corrad, M. Fanell, and L. Foschn. VM Consoldaton: a Real Case Based on OpenStack Cloud. Future Generaton Computer Systems, In Press. Fg. 2 Load balancng between nodes n the cloud system 5. Concluson Thus, the effectveness evaluaton of the algorthm for control of vrtual resources of the cloud system shows a performance boost from 12 to 15% compared to the standard. Our algorthm s very effectve for hgh-ntensty requests

7 Эффективное использование ресурсов распределенной платформы облачных вычислений для обеспечения качества мультимедийных услуг Trudy ISP RАN [The Proceedngs of ISP RAS], vol. 27, ssue 3, [2]. S. J. E. C. I. C. Clark, K. Fraser and A. Warfeld, "Lve mgraton of vrtual machnes, " In Proc. NSDI, [3]. N. Bobroff, A. Kochut, and K. Beaty, "Dynamc placement of vrtual machnes for managng SLA volatons," n Integrated Network Management, IM'07. 10th IFIP/IEEE Internatonal Symposum on. IEEE, 2007, pp [4]. Q. Huang, S. Su, S. Xu, J. L, P. Xu, and K. Shuang, "Mgraton-based elastc consoldaton schedulng n cloud data center," n Proceedngs of IEEE ICDCSW [5]. A scalable nfrastructure for CMS data analyss based on OpenStack Cloud and Gluster fle system S Toor et al 2014 J. Phys.: Conf. Ser [6]. Corrad, M. Fanell, and L. Foschn. VM Consoldaton: a Real Case Based on OpenStack Cloud. Future Generaton Computer Systems, In Press. И.П.Болодурина <prmat@mal.osu.ru>, Д.И. Парфёнов<fdot_t@mal.osu.ru> Оренбургский государсвенный университет, ,Россия, Оренбург, пр. Победы, д. 13 Аннотация. Проводимое исследование направлено на повышение эффективности использования высоконагруженных информационных систем, развернутых в облачной системе. Для этого планируется разработать модели, описывающие основные особенности обслуживания потоков с учетом топологий системы, сетевых сервисов и существующих систем планирования задач, а также методы управления потоками данных между процессами вычислительных задач. В рамках данной статьи решается задача исследования облачной системы и оценка эффективности схем управления с учетом различных алгоритмов планирования. С этой целью разработаны: модель облачной системы, метрики эффективности и методика экспериментального исследования алгоритмов планирования и методов управления потоками данных. Модели определяют функционалы вычислительных узлов и связанных потоков между сервисами всей системы в целом. Методика экспериментального исследования предполагает оценку эффективности совместной работы виртуальных машин с учетом алгоритмов планирования и методов управления потоками данных по описанным метрикам. Предложенные в рамках данной статьи решения являются основой разработанного симулятора облачной системы. Keywords: облачные вычисления, облачные системы, вычислительные узлы, вычислительный ресурсы, высоконагруженные информационные системы, балансировка нагрузки, качество мультимедийных услуг, виртуальные машины, виртуальные компоненты. Список литературы [1]. Qnga Huang, Ka Shuang, Peng Xu, Jan L, Xu Lu, Sen Su Predcton-based Dynamc Resource Schedulng for Vrtualzed Cloud Systems Journal of Networks, Vol 9, No 2 (2014), , Feb