Multi-class Multi-Server Threshold-based Systems: a. Study of Non-instantaneous Server Activation

Transcription

1 Mult-class Mult-Server Threshold-based Systems: a Study of Non-nstantaneous Server Actvaton 1 Cheng-Fu Chou, Leana Golubchk, and John C. S. Lu Abstract In ths paper, we consder performance evaluaton of a system whch shares K servers (or resources) among N heterogeneous classes of workloads, where server allocaton and de-allocaton for class s dctated by a class specfc threshold-based polcy wth hysteress control. In partcular, the server actvaton tme for class s non-nstantaneous. There are many systems and applcatons where a mult-class threshold-based queueng system can be of great use. One mportant utlty of usng threshold-based approaches s n stuatons where applcatons may ncur server usage costs. In these cases, one needs to consder not only the performance aspects but also the resultng cost/performance rato. The motvaton for usng hysteress control s to reduce the unnecessary cost of server setup (or actvaton) and server removal (or deactvaton) whenever there are momentary fluctuatons n workload. Moreover, servers n such systems and applcatons are often needed by multple classes of workloads, and hence, t s desrable to fnd good approaches to sharng server resources among the dfferent classes of workloads, preferably wthout statcally parttonng the server pool among these classes. An mportant and dstngushng characterstc of our work s that we consder the modelng and analyss of a mult-class system wth non-nstantaneous server actvaton, whch s of use n studyng many mportant applcatons. The man contrbutons of ths work are (a) n developng an effcent approxmaton method for solvng such models, (b) n verfyng the convergence of our teratve method, and (c) n evaluatng the resultng accuracy of the technque for computng performance measures of nterest, whch can subsequently be used n makng system desgn choces. I. INTRODUCTION In ths paper, we consder performance evaluaton of a mult-class mult-server system, n whch K servers are shared among N heterogeneous classes of workloads and K N. In ths mult-class multserver system, servers (resources) are needed by multple classes of workloads (applcatons) and we also note that not all classes of requests are operatng at hgh load at the same tme. That s, when the traffc loadng of class s low, t s not desrable to operate unnecessarly many servers for that class, due to the ncurred usage costs as well as due to the performance consequences of that class under-utlzng the servers whle other classes are (possbly) experencng hgh traffc workloads. On the other hand, C. F. Chou ccf@cse.ntu.edu.tw s wth Natonal Tawan Unversty. and L. Golubchk leana@cs.usc.edus wth Unversty of Southern Calforna and J. C. S. Lu cslu@cse.cuhk.edu.hk s wth Chnese Unversty of Hong Kong. Contact author L. Golubchk. Research has been funded n part by the NSF ANI grant (part of the jont NSF/CNPq program), by the RGC and the CUHK Manlne Research Grant, as well as by the Integrated Meda Systems Center, a Natonal Scence Foundaton Engneerng Research Center, Cooperatve Agreement No. EEC Any Opnons, fndngs and conclusons or recommendatons expressed n ths materal are those of the author(s) and do not necessarly reflect those of the Natonal Scence Foundaton.

2 2 t s also not desrable for a system to exhbt very long delays, whch can result from lack of servers under heavy loads. Therefore, t s an mportant ssue for the system to fgure out a good approach to use few resources to serve those requests from all N classes and stll attan a good cost/performance rato nstead of statcally parttonng the server pool among the classes. To deal wth the above ssue and effcently utlze system resources for such mult-class mult-server system, we propose a thresholdbased approach to dynamcally assgn servers to servces of dfferent class requests,.e., how to allocate or de-allocate servers to dfferent classes s governed by a set of thresholds. In ths work we use multclass mult-server threshold-based system to refer to our proposed threshold-based system whch s able to adaptvely share the servers among dfferent workloads wthout statcally parttonng the server pool among the classes. The motvaton for usng a threshold-based approach n a system s that applcatons may ncur server usage costs. Thus, one not only needs to consder the performance but also the cost/performance rato. One approach to mprovng the cost/performance rato of a mult-class mult-server system s to dynamcally react to changes n workload through the use of thresholds. For nstance, one can mantan the expected response tme of an applcaton at an acceptable level and at the same tme, mantan an acceptable cost for operatng that system by dynamcally addng or removng servers dependng on the traffc loadng. To possess the above property, one can use the threshold-based server allocaton approach to reduce the senstvty of performance characterstcs of a class of customers to the workload of other classes wthout havng to statcally partton resources between the classes. Note that n many cases, a smple threshold-based system may not suffce snce t s prone to workload oscllatons. One reason for avodng oscllatons n the above mentoned system s that there may be server setup and removal costs. Such workload oscllatons coupled wth non-neglgble server setup and removal costs can result n a poor cost/performance rato of a system. Ideally, one wants to add servers only when a system s movng toward a heavly loaded operaton regon, and one wants to remove servers only when a system s movng to-wards a lghtly loaded operaton regon t s not approprate to alter the number of servers durng momentary and small changes n workload. Such oscllaton behavor can be avoded by addng hysteress behavor. Hence the motvaton of ths work s lookng for effcent analyss technques of threshold-based queueng systems wth hysteress control. There are many applcatons where threshold-based resource management polces can be employed, and thus performance evaluaton of such systems through analyss of mult-class threshold-based queueng systems wth hysteress control can be of great use. For example, the Novell fle server mantans a memory pool wheren a fracton of t s used for communcaton buffers and a fracton s used for fle buffers, where threshold-based polces are mplemented n order to make decsons about when to ncrease the number of network buffers and when to decrease t; the threshold values are based on per-

3 3 ceved packet losses due to ncreases n network traffc actvty. Smlarly, OS desgn has been movng toward mantanng a common buffer space pool that can be dynamcally managed between the varous I/O processes. Another example applcaton s server replcaton for dfferent classes of Internet servces for an overlay network. As the number of requests for a partcular class of servce ncreases, the number of servers needed to mantan an acceptable level of qualty-of-servce guarantees also ncreases. The use of a threshold-based approach can result n a cost-controlled creaton/deleton of servers based on the changes n the workload for a partcular class of request. Thus, the model presented n ths paper and ts effcent soluton wll be benefcal for many systems and applcatons. Now, we begn to gve an overvew of the mult-class mult-server threshold-based system, whch has a total of K servers. In partcular, the number of servers employed for servcng class customers, {1,...,N}, s governed by a forward threshold vector F = [F (1), F (2),..., F (K 1)] (where F (1)<F (2)< <F (K 1)) and a reverse threshold vector R =[R (1), R (2),..., R (K 1)] (where R (1)<R (2)< <R (K 1)), where K s the maxmum number of servers that can be allocated to serve class customers (.e., the system ncludes hysteress control). The servce tme of class customers s represented by an exponental random varable wth mean µ 1. The server actvaton tme for class s non-nstantaneous, and t s represented by an exponental random varable wth mean β 1. In general, µ µ j and β β j for, j {1,...,N}. Next, we explan how to allocate or de-allocate servers to classes n the system as follows. Intally, each class s allocated a mnmum of one server. When a class customer arrves to an empty system (.e., when there are no other class customers), ths newly arrved class request s served by a sngle server. Arrval of a class customer when there are already F (j) class customers n the system (wth j servers already allocated to serve class ), causes an attempt to allocate one addtonal server to class, where j = 1,...,K 1. Departure of a class customer whch leaves behnd R (j) class customers (wth j + 1 servers already allocated to ths class pror to ths departure event), causes a de-allocaton of a server from class, where j = 1,...,K 1. In other words, ths forces the return of a server, whch was earler allocated to class, back to the pool of free servers whch are avalable for allocaton to all classes of customers. Therefore, all N classes of applcatons share a common pool of K servers, wth dynamc allocaton of servers to classes governed by a set of thresholds wth hysteress behavor. Note that when N =1 K K, then the classes do not nterfere wth each other snce the total peak resource demand s less than or equal to the total number of resources n the system. Of course, a more nterestng and challengng case s when N =1 K > K. In other words, we want to nvestgate the performance of each class of workload when the number of common servers s less than the total peak resource demand of all classes. By takng advantage of the fact that not all classes are operatng at hgh load at the same tme, one may use fewer resources (than wth statc resource parttonng) to serve requests from all N classes and stll acheve a good

4 4 cost/performance rato. Here arses another challengng problem,.e., how to determne what are good values for these forward and reverse threshold vectors, whch are a functon of many factors, such as the server setup, usage, and removal costs, characterstcs of the arrval process and the servce rates, as well as the possble nteracton between the dfferent classes of workloads. The goal of ths work s to develop an effcent method for soluton of mult-class mult-server threshold-based queueng system wth hysteress behavor wheren the server actvaton s non-nstantaneous. The queston of optmal values for the threshold vectors s, n general, a dffcult problem and s outsde the scope of ths paper. On the other hand, we want to pont out that effcent model soluton technques can be of great use n evaluatng varous parameter settngs (such as the threshold values). Such analytcal models are especally useful at desgn tme, when the speed of evaluaton s key. Thus, we beleve that our effcent soluton method facltates accessble expermentatons for nvestgatng the qualty of varous threshold parameters. Gven the above motvaton for the use of threshold-based systems wth hysteress control, we present an effcent technque for solvng the correspondng analytcal models and computng varous performance measures of nterest, n the context of non-nstantaneous server actvaton. We begn wth a very bref survey of some of the exstng lterature on the topc. A two-server system s consdered n [14], [15], [21]. An approxmate soluton for solvng a degenerate form of ths problem (where all thresholds are set to zero) s presented n [7], [9]; an approxmate soluton for a system that employs (non-zero) thresholds s presented n [22] (but wthout hysteress). In [8], the authors solve a mult-server thresholdbased queueng system wth hysteress, usng the Green s functon method [6], [10], [11]. In [17] we gve a soluton of several forms of the sngle class, mult-server threshold-based queueng system wth hysteress usng stochastc complementaton [18]. Technques for computaton of bounds for performance measures of sngle class, mult-server threshold-based queueng systems wth hysteress and nonnstantaneous server actvaton are gven n [3]. Lastly, [4] provdes a soluton technque for mult-class, mult-server threshold-based system wth hysteress and nstantaneous server actvaton. In ths paper, we extend and generalze that work to non-nstantaneous server actvaton; the non-nstantaneous server actvaton can have a sgnfcant mpact on the system performance (as wll be llustrated n Secton V) and hence s an mportant model characterstc to consder. Specfcally, n ths work we consder and solve a mult-class, mult-server threshold-based queueng system wth hysteress control and nonnstantaneous server actvaton. The contrbutons of ths work are as follows. To the best of our knowledge, none of the works descrbed above gve an effcent analytcal soluton technque for analyzng ths model. Snce n many applcatons, dfferent types of workloads compete for a pool of resources where server actvaton tme s non-neglgble (e.g., t takes a non-zero tme to replcate and actvate a vdeo server n an overlay

5 5 network), we consder t an mportant and dstngushng characterstc of our work. In ths paper, we present an teratve soluton technque whch solves the mult-class model by breakng t up nto N sngle class models, coupled through a set of model parameters whch capture the nteracton between classes. We also llustrate the accuracy of our approach, whch effcently computes performance measures of nterest, through a set of numercal results. Furthermore, we gve an proof and dscusson about the convergence of our teratve method to solve the approxmate model. Ths s mportant snce we can get further understandng of our analytc model such that we could construct more precse and effcent analytc model for the mult-class mult-server system. We also beleve that the effcency and accuracy of our teratve approach provdes an mportant step n fndng optmal threshold values for a mult-class, mult-server threshold based system wth hysteress and non-nstantaneous server actvaton. Fnally, we note that a varety of teratve approaches have been used n numerous approxmaton technques (e.g., refer to [2]). For nstance, an teratve technque for a somewhat dfferent control scheme for dynamc resource sharng between multple classes s employed n [19], [20]. The remander of ths paper s organzed as follows. In Secton II we gve a detaled descrpton of our model. Secton III descrbes our teratve soluton approach for ths model. The convergence of the teratve method to solve the approxmate model s presented n Secton IV. The qualty of ths approach,.e., ts accuracy and utlty n system desgn and evaluaton s dscussed n Secton V through the use of numercal results. Fnally, our conclusons are gven n Secton VI. II. SYSTEM MODEL The Markovan model for our mult-class, mult-server threshold-based queueng system wth hysteress control has an nfnte state space whch can be descrbed as follows. There are K servers n the system wth K N where N s the total number of classes of customers. The servce tme of dfferent classes can be dfferent, and the servce tme requrements of a class customer are exponentally dstrbuted wth parameter µ. The customer arrval process s Posson wth rate λ, where wth probablty α an arrvng customer s of class and N =1 α = 1 and 1 N. Addton and removal of servers for servng customers of class s governed by the forward and the reverse threshold vectors F = [F (1), F (2),, F (K 1)] and R = [R (1), R (2),, R (K 1)] where F (j) < F (j + 1) for 1 j K 2, R (j) < R (j + 1) for 1 j K 2, and R (j) < F (j) for 1 j K 1. Note that, unlke n [4], [5], the actvaton of a server for class s non-nstantaneous where the server actvaton tme s exponentally dstrbuted wth mean β 1. As mentoned n Secton I, ths s motvated by the fact that n many applcatons addton of a new server takes a non-neglgble amount of tme. Each of these K servers s able to serve a customer of any class. Each class starts out wth one server and may attempt to obtan at most K servers. These servers are allocated for servce of class customers

6 6 and returned to the pool of avalable servers based on the number of class customers currently n the system (as stated more formally below). In general, N =1 K may be greater than, equal to, or less than K; although the more nterestng and challengng case s where N =1 K > K. We model ths system as a Markovan process M, usng two dfferent varatons. In the frst varaton, we constran the number of class customers when the number of servers allocated to class s less than K (we motvate ths varaton below). The Markovan model for ths varaton s referred to as M a. In the second varaton, we do not use such a constrant, and the Markovan model for ths varaton s referred to as M b. We now gve a more detaled descrpton of each of these models. A. M a wth constrant vector a The Markovan process M a, wth a constrant vector a, has the followng state space S a : S a = {(n 1, s 1, l 1,..., n N, s N, l N ) n 0, l {1,...,K }, s {1, 2,..., K }, = 1,...,N} N =1 l K, l s, F (l ) n F (l ) + a l, where n s the number of class customers n the system, s s the number of busy (or actve) servers currently servng class customers, and l s the number of servers allocated to class, not all of whch may currently be avalable for servce of class customers snce server actvaton process s non-nstantaneous. Upon an arrval of a class customer, f F (j) n F (j) + a j where a j 0 and j = l, the system attempts to allocate an addtonal server for servce of class customers, whch s possble only when the system has suffcent amount of resources,.e., f N =1 l < K. Note that n a system where N =1 K > K, t may not always be possble to allocate another server snce t s possble that all K servers may have already been allocated. In ths case, the arrvng class customer jons the queue of class requests as long as F (j) n < F (j) + a j (where a j 0 and j = l ). When n = F (j) + a j, the arrvng class customer s rejected by the system f there s no server avalable for allocaton to class (.e., f l = K). For correctness, we assume the followng constrant on all a j : F (j) + a j < F (j + 1) + a j+1 for = 1, 2,...,N and j = 1, 2,..., K 1. We also assume that a K = ; hence, we have no restrctons on queue length when the maxmum number of servers that may be needed by class have been allocated (.e., when l = K ). The lmtaton on queue length when l < K can be motvated by system desgn consderatons. For example, f the system reaches a pont where ts desgn dctates that another server be allocated for class workload but a server s not avalable, then one may assume that the system s temporarly overloaded and rejecton of customers s a reasonable approach to dealng wth overload condtons. Of course, a real system

7 7 wll also not have an nfnte queue length, when the maxmum number of servers (K ) for class has been allocated. In ths case, we may ether (1) use a fnte queue length model (.e., a K s fnte) and study the system s performance under a gven queue sze lmtaton, or (2) allow an nfnte queue length (.e., a K = ) and use the model to study queue length requrements of the correspondng system. Our soluton methodology (refer to Secton III) allows for ether type of a model, but for smplcty of exposton, n the remander of the paper we wll focus our dscusson on the nfnte queue verson (.e., where a j s fnte, for j = 1,...,K 1, and a K = ). We now gve a detaled descrpton and formal structure for the transtons of M a. The transtons correspondng to an arrval of a class customer fall nto one of the followng categores: C 1 : no need to allocate another server to class. The condtons for ths category could be () the number of class customers does not cross a correspondng forward threshold or () there are already K servers allocated to class or () there are no avalable servers n the system due to resource contenton among the dfferent classes. C 2 : a need to allocate another server to class. The condton for ths category s that the number of allocated server for class s less than K and there s an avalable server n the system and the number of class customers crosses a forward threshold. The formal structure for these arrval transtons s as follows. (n 1, s 1, l 1,...,n, s, l,..., n N, s N, l N ) λα > (n 1, s 1, l 1,...,n + 1, s, l,..., n N, s N, l N ) f C 1 (1) (n 1, s 1, l 1,...,n, s, l,..., n N, s N, l N ) λα > (n 1, s 1,...,n + 1, s, l ,n N, s N, l N ) f C 2 (2) where condtons C 1 and C 2 are C 1 = C 2 = ( ) ) ( N (l < K ) (n < F (l )) (l = K (l < K ) ( l j = K) (F (l ) n < F (l ) + a l ) N (l < K l < K j=1 j=1 ( ) F (l ) n F (l ) + a l. ) The transtons correspondng to a departure of a class customer fall nto one of the followng categores: C 3 : no need to deactvate a server. The condtons for ths category are ether () only one server s allocated to class or () the number of class customers does not drop below a reverse threshold. C 4 : a need to deactvate a server. The condton for ths category s that more than one server s allocated to class and the number of class customers drops below a correspondng reverse threshold. The formal structure for these departure transton s as follows. (n 1, s 1, l 1,..., n, s, l,...,n N, s N, l N ) s µ >

8 8 (n 1, s 1, l 1,...,n 1, s, l,...,n N, s N, l N ) f C 3 (3) (n 1, s 1, l 1,..., n, s, l,...,n N, s N, l N ) s µ > (n 1, s 1, l 1,...,n 1, mn(s, l 1), l 1,...,n N, s N, l N ) f C 4 (4) where condtons for C 3 and C 4 are C 3 = C 4 = ( ) ( ) (n > 0) (l = 1) (n > 0) (n 1>R (l 1)) (l >1) ( ) (n > 0) (n 1 = R (l 1)) (l > 1). Lastly, there are transtons correspondng to class server actvatons. The condton for these transtons s that the number of actve servers s less than the number of allocated servers. The formal structure for these actvaton transtons s as follows. (n 1, s 1, l 1,...,n, s, l,..., n N, s N, l N ) (l s )β > (n1, s 1, l 1,..., n, s + 1, l,..., n N, s N, l N ) f C 5 (5) where condton C 5 = (l > s ). B. M b wthout constrant a The second model varaton s M b, whch represents a Markovan process wthout constrants on the number of class customers. It has the followng state space, S b : S b = {(n 1, s 1, l 1,...,n N, s N, l N ) n 0, l {1,...,K }, l s, s {1,..., K }, s K, = 1,...,N} where n s the number of class customers n the system, s s the number of busy (or actve) servers currently servcng class customers, and l s the number of servers expected to be allocated/actvated for class use more specfcally, accordng to the threshold vectors, l servers should be n use by class customers but may not be, because () multple classes of customers are competng for these servers and () n our model server actvaton s non-nstantaneous. Hence, a major dfference between M a and M b s that we use a constrant vector a to lmt the queue length when l < K n M a whle we do not lmt the queue length n M b. We gve a comparson study between these two models n Secton V. We now gve a detaled descrpton and formal structure for the transtons of M b. The transtons correspondng to arrvals of class customers fall nto one of the followng categores: C 1 : no need to ncrease the number of expected servers (accordng to the forward threshold vector) for class. The condtons for ths category could be () the number of class customers does not cross a correspondng forward threshold or () the number of expected servers s equal to K.

9 9 C 2 : a need to ncrease the number of expected servers for class. The condton for ths category s that the number of expected servers for class s less than K and the number of class customers crosses a forward threshold. The formal structure for these arrval transtons s as follows. (n 1, s 1, l 1,...,n, s, l,...,n N, s N, l N ) λα >(n 1, s 1, l 1,..., n + 1, s, l,..., n N, s N, l N ) f C 1 (6) (n 1, s 1, l 1,...,n, s, l,...,n N, s N, l N ) λα >(n 1, s 1,..., n + 1, s, l ,n N, s N, l N ) f C 2 (7) where condtons C 1 and C 2 are C 1 = (l < K ) (n < F (l )) l = K and C 2 = l < K F (l ) = n. The transtons correspondng departures of class customers fall nto one of the followng categores: C 3 : no need to deactvate a server. The condton for ths category s that ether () the system has only one server for class or () the number of class customers does not drop below a reverse threshold. C 4 : a need to deactvate a server. The condton for ths category s that there s more than one server for class and the number of class customers drops below a correspondng reverse threshold. The formal structure for these departure transtons s as follows. (n 1, s 1, l 1,...,n, s, l,..., n N, s N, l N ) s µ > (n 1, s 1, l 1,...,n 1, s, l,..., n N, s N, l N ) f C 3 (8) (n 1, s 1, l 1,...,n, s, l,..., n N, s N, l N ) s µ > (n 1, s 1, l 1,..., n 1, mn(s, l 1), l 1,...,n N, s N, l N ) f C 4 (9) where condtons C 3 and C 4 are C 3 = C 4 = ( ) ( ) (n > 0) (l = 1) (n > 0) (n 1 > R (l 1)) (l > 1) ( ) (n > 0) (n 1 = R (l 1)) (l > 1). Lastly, there are the transtons correspondng to class server actvaton. The condton for these transtons s that () the number of actve servers s less than the number of expected servers and () there

10 10 s an avalable server n the system. The formal structure for these server actvaton transtons s as follows. (n 1, s 1, l 1,...,n, s, l,...,n N, s N, l N ) β > (n 1, s 1, l 1,..., n, s + 1, l,...,n N, s N, l N ) f C 5 (10) where the condton C 5 = ( (l > s ) ( N j=1 lj < K) ). III. ITERATIVE METHOD In ths secton we descrbe an teratve approach to solvng the models presented n Secton II. As descrbed n Secton II, the correspondng Markov process 1, M, s nfnte n multple dmensons. One can choose to solve ths model by (a) smulatng the Markovan process M, or (b) lookng for specal structure, or (c) lookng for effcent approxmaton technques. Because M appears to lack suffcent structure for an effcent exact soluton technque (e.g., such as the matrx-geometrc technque), we descrbe an approxmate teratve soluton technque for solvng ths model. The use of an approxmaton s motvated by the desre to construct an effcent soluton approach (and smulaton can be sgnfcantly slower than analytcal solutons) as well as an accurate one (and teratve technques can often produce farly accurate results). A. Basc Approach Let us frst descrbe the basc approach to solvng the above defned Markovan model. We frst break up the orgnal model M nto N sngle class Markovan sub-models, namely, M 1, M 2,..., M N (see Secton III-B for a more detaled descrpton of the M s). These N Markovan models are coupled va a set of blockng probabltes. Specfcally, the nteracton between classes occurs when class requres allocaton of another server (due to the crossng of a forward threshold), and no servers are avalable n the system (.e., all K servers have already been allocated) due to the workload of other classes. Therefore, n general, there s a non-zero probablty that class, whch has already (a) allocated s servers n the case of M a or (b) expected to be allocated/actvated s servers n the case of M b, s not able to add a server upon the forward threshold crossng. Let us refer to ths as a blockng probablty P,s, whch approxmately captures ths nteracton between classes. Note that, s = l (number of allocated servers for class ) n M a whle s = l (number of expected to be allocated/actvated servers for class ) n M b. We now formally descrbe our teratve approach. Let M (n) be the Markovan process correspondng to the ndvdual class model at teraton n wth a correspondng steady state probablty vector π (n). The parameters of each M (n) are computed as a functon of blockng probabltes, P (n) = 1 In the remander of the paper, we use M to represent ether M a or M b, for smplcty of exposton.

11 11 {P (n),1, P (n),2,...,p (n),k }, whch are n turn computed as a functon of the steady state probablty vector, π (n 1), obtaned durng the prevous teraton. (We gve the detals of the constructon of M (n) and the computaton of π (n) below 2.) Then, a hgh level descrpton of our teratve approach s as follows (a more detaled and formal descrpton s gven n Secton III-C): 1. Construct M (0) 1, M (0) 2,...,M (0) N ; set n = 0 (ths s teraton 0); 2. Solve M (n) 1, M (n) 2,..., M (n) N,.e., compute the correspondng steady state probabltes to obtan π (n) 1, π (n) 2,..., π (n) ; set n = n + 1; N 3. Use these steady state probabltes to compute P (n) 1, P (n) 2,..., P (n) N ; 4. Use these blockng probabltes to update the ndvdual class models,.e., construct M (n) 1, M (n) 2,..., M (n) N, where for each = 1,...,N, parameters of M (n) are computed as functons of P (n) (but not P (n) j where j ); 5. Contnue the teratve process (.e., go back to step 2) untl the values of all P s converge. B. Indvdual Class Model Snce our teratve approach nvolves soluton of ndvdual class models (M s) we now brefly descrbe the class model, whch can be defned as follows. We have K servers each wth an exponental servce rate µ. Customer arrvals are governed by a Posson process wth rate = α λ. Addton and removal of servers s governed by the forward and the reverse threshold vectors, namely F = [F (1), F (2),...,F (K 1)] and R = [R (1), R (2),..., R (K 1)]. where R (j) < F (j) and 1 j K 1. And, server actvaton tme s exponentally dstrbuted wth rate β. Indvdual Class Model for M a Gven a K -server sngle class threshold-based queueng system wth hysteress control and constrant vector a, we model t as a Markov process M wth the followng state space S : S = {(k, j, l) k 0; j, l {1, 2,..., K }, l j, F (l) k < F (l) + a l } where k s the number of customers n the class queueng system, j s the number of busy (actve) servers, and l s the number of allocated, not all of whch may currently be actvated due to the nonnstantaneous nature of server actvaton n our model. Fgure 1 llustrates the state transton dagram for such a system where K = 2. Formally, the transton structure of M can be specfed as n Table I 3, where all transtons are from state (k, j, l), wth the state descrpton gven above: 2 Note that there are multple approaches to constructng M (0) s,.e., multple ways to start the teraton; we gve detals of one such approach below. 3 Note that, the transton rates descrbed here are a functon of the blockng probabltes, P,l, whch change from teraton to teraton, as outlned above; however, for smplcty of notaton, we do not ndcate the teraton step number n the descrpton of the transton structure of a class model.

12 12 λ P,1 P,1 0,1,1 1,1,1 µ... F (1),1 µ,1 µ F (1)+1,1,1... µ F (1)+a 1 1,1 µ (1-P,1 ) (1-P,1 ) (1-P...,1 ) 2µ R (1)+1,1,2 R (1)+2,1,2 µ µ... F (1)+1 F (1)+2... F 1 (1)+a F (1)+a 1...,1,2,1,2 +1,1,2 +2,1,2 µ µ µ µ µ β β β β β β R (1)+1,2,2 2µ R (1)+2,2,2... F (1)+1 F (1)+2... F 1 (1)+a F 1 (1)+a... 2µ,2,2,2,2 +1,2,2 +1,2,2 2µ 2µ 2µ 2µ 2µ Fg. 1. State transton dagram of M a for a class system wth K = 2. Next State Rate Condton for transton (k + 1, j, l) (1 l < K ) (k < F (l)) (k + 1, j, l) l = K (k + 1, j, l) P,l (1 l < K ) (F (l) k < F (l) + a l ) (k + 1, j, l + 1) (1 P,l ) (1 j < K ) (F (l) k < F (l) + a l ) (k 1, j, l) jµ (k 1) (1 < l K ) (k 1 > R (l 1)) (k 1, mn(j, l 1), l 1) jµ (k 1) (1 < l K ) (k 1 = R (l 1)) (k 1, j, l) µ (l = j = 1) (k 1) (k, j + 1, l) (l j)β (l > j) TABLE I DESCRIPTION OF STATE TRANSITION FOR M a Indvdual Class Model for M b Gven a K -server sngle class threshold-based queueng system wth hysteress control, we model t as a Markov process M wth the followng state space S : S = {(k, j, l) k 0; j, l {1, 2,..., K }, l j} where k s the number of customers n the class queueng system, j s the number of busy (actve) servers, and l s the number of servers expected to be allocated and actvated. Fgure 2 llustrates the state transton dagram for such a system where K = 2. Formally, the transton structure of M can be specfed as n Table II, where all transtons are from state (k, j, l), wth the state descrpton gven above: Let us now proceed to a more detaled descrpton of the teratve soluton technque for the mult-class system. We do ths under the assumpton that, gven P, we know how to construct M (usng Table I or Table II above) and compute π, the steady state probablty vector correspondng to M. The procedure

13 13 0,1,1 1,1,1 µ µ... F (1),1,1 µ 2µ R (1)+1,1,2 R (1)+2,1,2 µ µ... F (1)+1 F (1)+2... F 1 (1)+a F (1)+a 1...,1,2,1,2 +1,1,2 +2,1,2 µ µ µ µ µ (1 P,1 )β (1 P,1 )β (1 P,1 )β (1 P,1 )β (1 P,1 )β (1 P,1 )β R (1)+1,2,2 2µ R (1)+2,2,2... F (1)+1 F (1)+2... F 1 (1)+a F 1 (1)+a... 2µ,2,2,2,2 +1,2,2 +1,2,2 2µ 2µ 2µ 2µ 2µ Fg. 2. State transton dagram of M b for a class system wth K = 2. Next State Rate Condton (k + 1, j, l) (1 l<k ) (k<f (l)) (k + 1, j, l) (l = K ) (k + 1, j, l + 1) (1 j < K ) k = F (l) (k 1, j, l) jµ (k 1) (1 < l K ) (k 1 > R (l 1)) (k 1, mn(j, l 1), l 1) jµ (k 1) (1 < l K ) (k 1 = R (l 1)) (k 1, j, l) µ (l = j = 1) (k 1) (k, j + 1, l) (1 P,j )β (l > j) TABLE II DESCRIPTION OF STATE TRANSITION FOR M b for computng π, s gven n Secton III-E. C. Iteratve Computaton In ths subsecton, we descrbe the framework for the teratve procedure. Ths teratve procedure s smlar to our work n [4] but we extend t to handle the case wheren the server actvaton event s non-nstantaneous. Frst, note that n general, there are two cases to consder here: case 1: N =1 K K; that s, we have a trval case, where the classes do not nterfere wth each other, and we can solve each ndvdual class model once (.e., no need for teraton) usng the procedure gven n Secton III-E wth P,s = 0,, s. case 2: N =1 K > K, where t s possble that an attempt at server allocaton for class may fal because all K servers n the system are currently allocated. As descrbed above, n ths case a form of blockng occurs and we solve the model usng the teratve approach outlned n Secton III-A whose detals are now presented below.

14 14 Note also that the man dffculty n the teratve technque outlned n Secton III-A s n determnng an approprate procedure for computng the blockng probabltes whch capture the class nteracton,.e., the probabltes that, upon a forward threshold crossng, t s not possble to allocate another server to class. Recall that, durng the n th teraton (n 0), P (n),s s the blockng probablty of class (1 N) to whch s servers already have been (a) allocated n the case of M a or (b) expected to be allocated/actvated n the case of M b. Before we proceed, let us state the followng defntons. Defnton 1: Let X and Y be two non-negatve random varables havng values n {1, 2,...} and let π X and π Y be ther respectve probablty mass functons. Let Z be another non-negatve random varable where Z = X + Y; then π Z = π X π Y where s the convoluton operator. Defnton 2: Let X be a non-negatve random varable havng values n {1, 2,..., } and let π X be ts probablty mass functon. Let X = { X f L 1 X L 2 0 otherwse. Then the probablty mass functon of X, denoted by π X, s equal to g(π X, L 1, L 2 ) where functon g s defned such that: π X[k] L2 f L 1 k L 2 π g(π X, L 1, L 2 )[k] = π X [k] = m=l X[m] 1 0 otherwse (11) Let π (n) [k, j, l] be the steady state probablty of class havng k customers (k 0) n the system wth j actvated servers and l target server allocatons (wth 1 j l K ), computed durng the n th teraton. Let π (n) denote the steady state probablty vector of the number of servers allocated to class, where π (n) [l] denotes the steady state probablty of l servers havng been target allocated to class, as computed durng the n th teraton. Thus, we have: π (n) [l] = K j=1 k π (n) [k, j, l] (12) Fnally, let Q (n) be the transton rate matrx correspondng to the class model M (n), durng the n th teraton, whch s computed usng the transton structure of M (n) gven n (a) Table I n the case of M a or (b) Table II n the case of M b, and P (n 1),s, where 1 s K 1. Then, the teratve procedure s as follow: 1. Intalzaton step: set n = 0 and set P (0),s = 0 for 1 s < K. Gven these ntal values of blockng probabltes, for each class, we can construct Q (0) the case of M a or (b) Table II n the case of M b, and then compute π (0) usng the transton structure gven n (a) Table I n usng the procedure gven n

15 15 Secton III-E. Once we compute the steady state probablty vector π (0) for each class, we can then compute ther respectve server allocaton probablty vectors, π (0) s, usng Equaton (12). The π (0) s are n turn needed n the computaton of the blockng probabltes, P (1),s s (step 2 below). 2. Updatng of blockng probabltes step: n = n + 1, and P (n),s = 0 f K N j=1 K j 0 f K s > N j=1,j K j Γ(, s, n) otherwse (13) The frst condton n Equaton (13) ndcates that the system has a suffcent number of servers for all classes (we nclude ths for completeness). The second condton ndcates that the system has suffcent resources to allocate at least one more server to class wthout affectng the maxmum possble server allocaton of other classes. In the last condton, the Γ functon s used to compute the blockng probablty, at teraton n, for class whch has s servers already (a) allocated to t n the case of M a or (b) expected to be allocated/actvated n the cased of M b. Γ(, s, n) can be computed as follows. Let A m (, s, n) be the random varable, at teraton n, denotng server allocaton of class m, when class has been (a) allocated s servers n the case of M a or (b) expected to be allocated/actvated s servers n the case of M b. Let Υ m (, s, n) be the probablty mass functon of A m (, s, n). Then we have: Υ m (, s, n) = g(π (n 1) m, 1, L m ) (14) for m = {1, 2,..., 1, + 1,..., N} where functon g s defned through Equaton (11) and L m s as follows: L m = { Km f K s (N 2) K m K s (N 2) otherwse (15) and π (n 1) m n Equaton (14) s computed usng Equaton (12). The normalzaton n Equaton (14) s used to account for the fact that f we know that the system already (a) allocated s servers to class n the case of M a or (b) expected to be allocated/actvated s servers to class n the case of M b, then the system only has (K s ) servers remanng. Out of these (K s ) remanng servers, the system needs to allocate (N 2) to customers that are nether n class nor n class m (.e., the system allocates at least one server to each class). Therefore, f the system potentally has at least K m avalable servers, then A m (, s, n) can have values n {1,...,K m }; otherwse, the random varable A m (, s, n) can only take on values n {1, 2,..., K s (N 2)}. Let B(, s, n) be a non-negatve random varable, at

16 16 teraton n, denotng the server allocaton of all classes except class, where class already has s servers (a) allocated to t n the case of M a or (b) expected to be allocated/actvated n the case of M b. Let Ψ(, s, n) be the probablty mass functon of B(, s, n). Then we have: Ψ(, s, n) = g ((Υ 1 (, s, n) Υ +1 (, s, n) Υ N (, s, n)), N 1, K s ) (16) The normalzaton n Equaton (16) s used to account for the fact that f the system has already (a) allocated s servers to class n the case of M a, or (b) expected to be allocated/actvated s servers to class n the case of M b, then the number of servers that have been allocated to other classes can only range n {N 1, N,..., K s }. Lastly, Γ(, s, n), the functon used to compute blockng probabltes, at teraton n, correspondng to class wth (a) s allocated servers n the case of M a or (b) s expected to be allocated/actvated servers n the case of M b s: Γ(, s, n) = Ψ(, s, n; K s ) (17) where Ψ(, s, n; K s ) = Prob[B(, s, n) = K s ] and Ψ(, s, n) s computed usng Equaton (16). 3. Updatng of ndvdual class models step: gven the blockng probabltes P (n),s of class n Equaton (13), we can compute the new rate matrx Q (n) (based on the transton structure gven n (a) Table I n the case of M a or (b) Table II n the case of M b ) and then compute the correspondng steady state probabltes π (n) (usng the procedure gven n Secton III-E) as well as π (n), the probablty vector of server allocaton of class (usng Equaton (12)). (The π (n) s wll n turn be needed n the updatng of the blockng probabltes, P (n+1),s s (step 2 above).) 4. Test of convergence step: f P (n),s P (n 1) ɛ for each class, 1 N, and each s, 1 s K 1, then stop. Otherwse, go to step 2 and contnue teratng. D. Computaton of Performance Measures,s In ths secton we brefly dscuss computaton of performance measures. Due to lack of space, we only present the dervaton for model M a and we could use the same approach for the dervaton for model M b. Gven the steady state probabltes π, = 1,..., N, computed usng the teratve approach descrbed above, we can compute varous performance measures of nterest. More specfcally, for each class we can compute performance measures whch can be expressed n the form of a Markov reward functon, R, where R = k,j,l π [k, j, l]r (k, j, l) and R (k, j, l) s the reward for state (k, j, l) of class. Some useful performance measures nclude: (a) expected number of customers of class, (b) expected response tme for customers of class, (c)

17 17 probablty of droppng a customer of class upon ts arrval, (d) throughput of class customers, and so on. For nstance, let E[N ] and E[T ] denote the expected number of customers and the expected response tme, respectvely, of the class model, correspondng to the Markov process M. Then E[N ] can be expressed as k,j,l k π [k, j, l]. (A more detaled expresson for E[N ] s gven n equaton (33) n Appendx A.) Of course, usng Lttle s result [16], we have E[T ] = 1 λ E[N ], where λ throughput. To compute λ s the class we need to account for the customers that are dropped from the system (see Secton II). Hence, λ = (1 K l=1 lj=1 P,j π [F (j) + a j, j, l]). We beleve that the more nterestng performance measures are those computed on a per class bass, snce a useful part of studyng performance of mult-class threshold-based systems s to dscover the effect that the varous classes have on one another. Therefore, we have concentrated on per class performance measures here. However, we can also use these to compute overall system performance measures, for nstance, as a weghted average of the ndvdual class performance measures. For example, we can compute the expected system response tme, E[T], as follows: where λ = N =1 λ. E. Analyss of the Indvdual Class Model E[T] = λ 1 λ E[T 1] + λ 2 λ E[T 2] + + λ N λ E[T N] In ths secton we brefly summarze the soluton technque for the ndvdual class model whch was defned n Secton III-B. Specfcally, we use the sngle class soluton technque we derved n [17] wth some modfcatons needed to account for the structure of the mult-class model. Snce these modfcaton are mostly straghtforward, we only summarze the soluton technque n ths secton, and gve the detals n Appendx A for completeness. The general approach s as follows. As already stated, we model the class queueng system as a Markov process, M, where: (1) the man goal s to compute the steady state probabltes of the Markov process and use these to compute varous performance metrcs of nterest and (2) the man dffculty s that the Markov process s nfnte (see Secton III-B) and thus dffcult to solve usng a drect approach 4. As s often done n these cases, we need to look for specal structure that mght exst n the Markov process; specfcally, we take advantage of the stochastc complementaton technque [18]. The basc approach to computng the steady state probabltes of the Markov process and the correspondng performance measures s as follows. Frst, we construct an upper bound model, M u, for the orgnal Markov 4 We could consder fnte versons of the model or truncaton of the nfnte verson [12]; however, n ether case the Markov process would stll be very large and the computatonal complexty of a drect soluton for a reasonable sze system stll hgh.

18 18 process M, whle tryng to satsfy the crtera that the new model wll: (1) provde (hopefully a tght) upper bound on the desred performance measures and (2) be a smpler model to solve. Therefore, the upper bound model transtons that replace the orgnal transton can be specfed as follows: In Tables I and II we replace (k 1, mn(j, l 1), l 1) wth (k 1, 1, l 1), where the transton rate s jµ. Next, we partton the state space of the orgnal Markov process M 5 nto dsjont sets. Usng the concept of stochastc complementaton, for each set, we compute the condtonal steady state probablty vector, gven that the orgnal Markov process M s n that set. (A relatvely smple constructon of the stochastc complement s possble due to the specal structure that exsts n the ndvdual class models; specfcally we explot the sngle entry structure as n [3].) By applyng the state aggregaton technque [1], we aggregate each set nto a sngle state and then compute the steady state probabltes for the aggregated process,.e., the probabltes of the system beng n any gven set. Lastly, we apply the dsaggregaton technque [1] to compute the ndvdual (uncondtonal) steady state probabltes of the orgnal Markov process M. These can n turn be used to compute varous performance measures of nterest. (Refer to Appendx for a detaled dervaton of the soluton of M.) IV. CONVERGENCE OF THE ITERATIVE METHOD As descrbed n Secton III, we break up the orgnal model M nto N sngle class Markovan submodels,.e., M 1, M 2,...,M N. We use a set of blockng probabltes to descrbe the nteracton among these N Markovan sub-models. In partcular, the blockng probabltes P for class are functons of all steady state probabltes except the steady state probablty of class,.e., π. Therefore, we could use N groups of smultaneous equatons to represent those N Markovan sub-models snce the blockng probabltes could be represented by the functon of those steady state probabltes. Note that for each class the number of unknown varables s π and there are N =0 π varables for the whole system. Let A be the transton rate matrx for the class except all the entres n the frst column are equal to 1. In addton, b s a row vector, n whch all the elements are 0 but the frst element s 1, and ts sze s the same as π. Thus, we can use the followng non-homogeneous system of (non-lnear) equatons to represent the class Markovan sub-model M : π A = b. To use the teratve method to solve N groups of smultaneous equatons, we splt the matrx A nto two sub-matrces A,n and A,b, where A,n s the ntal nonsngular, non-blockng matrx (wth all blockng probabltes are equal to 0) and A,b = A A,n. Gven a splttng wth nonsngular A,n, we have π (A,n + A,b ) = b, π = b A 1,n π A,b A 1,n (18) 5 For smplcty, we use M nstead of M u n the rest of ths paper.

19 19 whch leads to our teratve procedure. π k = b A 1,n π k A k,ba 1,n. (19) We note that A k,b s updated based on the steady state probabltes πk 1 j, where 1 j N and j, n the prevous k 1 round. From the above dscusson, one can compute the error vector of class, e.g., e k after the kth round, we have: e 0 e 1 e 2 = π 0 π = b A 1,n (b A 1,n π A,b A 1,n ) = πa,ba 1,n, (20) = π 1 π = π 1 A 1,bA 1,n π A,b A 1,n, (21) = π 2 π = π 2 A2,b A 1,n π A,b A 1,n (22). e k = π k π = π k Ak,b A 1,n π A,b A 1,n. (23) If our teratve method s able to converge, the suffcent condton s that e k becomes very close to 0 after fnte k teratons. Snce both π k and π are statonary probablty vectors, we can see that as A k,b approaches to the actual A,b, e k wll approach 0. To llustrate the above argument, we gve the proof of a 2-class system and show the convergence of the proposed teratve method. Example: Consder a 2-class, 3-server threshold-based system as follows: K = 3, K 1 = K 2 = 2, F 1, R 1, F 2, a and R 2. We assume that there exts a statonary dstrbuton for ths system. Let π,j be the steady state probablty for class wth j servers and then π 1 = ( π 1,1, π 1,2 ), and π 2 = ( π 2,1, π 2,2 ). Accordng to our method to compute the blockng probabltes, we have P 1,1 = π 2,2 and P 2,1 = π 1,2. Therefore, the goal s to solve two groups of non-homogeneous of lnear equatons,.e., π 1 A 1 = b 1 and π 2 A 2 = b 2, where P 1,1 = π 2,2 and P 2,1 = π 1,2. Step 0: we set P1,1 0 = 0 and P0 2,1 = 0. P 0 1,1 = 0 < P 1,1 π 0 1,2 > π 1,2 = P 2,1 and π 0 1,1 < π 1,1 P 0 2,1 = 0 < P 2,1 π 0 2,2 > π 2,2 = P 1,1 and π 0 2,1 < π 2,1 Step 1: we set P 1 1,1 = π 0 2,2 and P 1 2,1 = π 0 1,2. P 1 1,1 > P 1,1 π 1 1,2 < π 1,2 = P 2,1 and π 1 1,1 > π 1,1 P 1 2,1 > P 2,1 π 1 2,2 < π 2,2 = P 1,1 and π 1 2,1 > π 2,1. Step 2: we set P 2 1,1 = π1 2,2 and P2 2,1 = π1 1,2. P 2 1,1 < P 1,1 π 2 1,2 > π 1,2 = P 2,1 and π 2 1,1 < π 1,1 P 2 2,1 < P 2,1 π 2 2,2 > π 2,2 = P 1,1 and π 2 2,1 < π 2,1.

20 20 Step 3: we set P 3 1,1 = π 2 2,2 and P 3 2,1 = π 2 1,2. P 3 1,1 > P 1,1 π 3 1,2 < π 1,2 = P 2,1 and π 3 1,1 > π 1,1 P 3 2,1 > P 2,1 π 3 2,2 < π 2,2 = P 1,1 and π 3 2,1 > π 2,1. Step 2k: we set P1,1 2k = π2k 1 2,2 and P2,1 2k = π2k 1 1,2. P 2k 1,1 < P 1,1 π 2k 1,2 > π 1,2 = P 2,1 and π 2k 1,1 < π 1,1 P 2k 2,1 < P 2,1 π 2k 2,2 > π 2,2 = P 1,1 and π 2k 2,1 < π 2,1. Step 2k+1: we set P1,1 2k+1 = π 2k 2,2 and P2k+1 2,1 = π 2k 1,2. P 2k+1 1,1 > P 1,1 π 2k+1 1,2 < π 1,2 = P 2,1 and π 2k+1 1,1 > π 1,1 P 2k+1 2,1 > P 2,1 π 2k+1 2,2 < π 2,2 = P 1,1 and π 2k+1 2,1 > π 2,1. Next, we would lke to prove the followng nequalty:. P,1 2k < P2k+2,1 < P,1 < P,1 2k+3 < P,1 2k+1, where = 1 or 2 and k 0. (24) One can use the mathematcal nducton to prove the above nequalty. 1. When k = 0, t s trval to verfy the nequalty s correct for class 1 or class Assume k = m, the nequalty holds for class 1 and class As k = m + 1, for class 1, we have P 2m 1 2,1 > P 2m+1 2,1 P 2m 1,1 = π 2m 1 2,2 < π 2m+1 2,2 = P 2m+2 1,1 ; that s, we use the hgher blockng probablty for class 2, we get lower state probablty π 2m 1 2,2, whch s the blockng probablty of class 1 for the next round. Smlarly, we can prove another sde of the nequalty for class 1: P2,1 2m < P2m+2 2,1 P1,1 2m+3 < P1,1 2m+1. Of course, the same approach can be used to prove the nequalty for class 2. Once we show that Equaton (24) holds, t s not dffcult to show that A k,b approaches A,b as k ncreases,.e., e k s close to 0. In the followng, we gve a proof of the convergence of the teratve method for a 2-class mult-server system. Lemma 1: Consder a 2-class, K-server threshold-based system as follows: K, K 1, K 2, F 1, R 1, F 2, a and R 2. Assume that there exts a statonary state dstrbuton for the system, the proposed teratve method for solvng ths system wll converge to the steady state dstrbuton. Proof: Let π,j be the steady state probablty for class wth allocated j servers, we have π 1 = ( π 1,1, π 1,2,, π 1,K1 ), and π 2 = ( π 2,1, π 2,2,, π 2,K2 ). Accordng to our method to compute the blockng probabltes, for class 1 we have P 1,1 = = P 1,K K2 1 = 0 and P 1,x = K K 2 x K 2. Smlarly, for class 2 we get P 2,1 = = P 2,K K1 1 = 0 and P 2,y = π 2,K x K x m=1 π 2,m, where π 1,K y K y m=1 π 1,m,