MAX-MIN OPTIMALITY OF SERVICE RATES IN QUEUEING SYSTEMS WITH CUSTOMER-AVERAGE PERFORMANCE CRITERION. Li Xia Ming Xie Wenjun Yin Jin Dong

Transcription

1 Proceedings of the 28 Winter Simulation Conference S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. MAX-MIN OPTIMALITY OF SERVICE RATES IN QUEUEING SYSTEMS WITH CUSTOMER-AVERAGE PERFORMANCE CRITERION Li Xia Ming Xie Wenjun Yin Jin Dong IBM China Research Laboratory Diamond Building, Zhongguancun Software Park Haidian District, Beijing 1193, P. R. CHINA ABSTRACT In this paper, we consider the optimization of service rates in queueing systems, especially in closed Jackson networks. The optimization criterion is the customer-average performance, which is another important performance metric compared with the traditional time-average performance. Based on the methodology of perturbation analysis, we can derive a performance difference equation when the service rates are changed. With this difference equation, we find the optimal service rates have a Max-Min property, i.e., the optimal service rates can be chosen from its maximal or minimal value. This property can reduce the complexity of this type of optimization problems. Moreover, we also prove the max-min optimality is valid for both state-dependent service rates and load-dependent service rates in queueing systems. 1 INTRODUCTION Perturbation analysis (PA) is an important theory for the performance optimization of discrete event dynamic systems (DEDS). It gives an efficient way to estimate the performance gradient with respect to system parameters (e.g., the service rates of servers in queueing systems) based on a single sample path (Cao 1994, Cassandras and Lafortune 1999, Glasserman 1991, Gong and Ho 1987, Ho and Cao 1991, Ho, Cao, and Cassandras 1983). With the estimated performance gradients, the gradient-descent algorithms can be used to optimize the system parameters (Chong and Zak 21). The traditional PA theory of queueing systems is extended to Markov systems in the past decade. With the PA theory of Markov systems (we also call it Markov performance potential theory), many new insights into Markov decision processes (MDP) are obtained from a new point of view (Cao and Chen 1997, Cao 23). Although the perturbations in queueing systems are continuous in nature and those in Markov systems are discrete, Xia and Cao (26b) establish the relationship between these two theories. It bridges the gap between PA theory in these two systems. Based on this relationship, we can develop the parallel results between these two theories, such as the performance difference equations and policy iteration algorithms to optimize the service rates under the customer-average performance criterion in queueing systems (Xia, Chen, and Cao 28). In most of the optimization algorithms, time-average performance is used as the performance metric of queueing systems. In fact, customer-average performance is also an important performance metric in queueing systems. It quantifies the system performance averaged by the number of served customers and reflects the idea of customeroriented, which is very popular in many service industries. In practice, many performance measurements belong to the customer-average performance, such as the average waiting time of each customer in a banking system, the probability of a packet successfully reaching the destination node in a wireless communication network, and so on. On the other hand, to the best of our knowledge, there seems not to exist any algorithm for the customer-average performance. Moreover, by a numerical experiment presented in this paper we find that the algorithm for time-average performance can not be directly used to optimize the customer-average performance. Therefore, it is desirable to study the optimization of service rates under the customer-average performance criterion. In this paper we consider the optimization of service rates in a closed Jackson network. The objective is to change the service rates to achieve the optimal customeraverage performance. We study the optimization of two types of service rates, state-dependent service rates and load-dependent service rates. With the performance difference equation in PA theory, we prove that the service /8/$ IEEE 59

2 rates have the Max-Min optimality when the performance function satisfies some properties. It means that we only need to choose the maximal or minimal values of service rates during the optimization of this type of problems. It can greatly reduce the complexity of optimization problems. The Max-Min optimality of service rates is similar to the results of Ma and Cao (1994), Yao and Schechner (1989). But in this paper the optimization criterion is customeraverage performance, which is different from the timeaverage performance discussed by Ma and Cao (1994), Yao and Schechner (1989). Moreover, the previous articles (Ma and Cao 1994, Yao and Schechner 1989) only discuss the load-dependent service rates. Here we prove the Max- Min optimality for both state-dependent service rates and load-dependent service rates. Furthermore, in this paper we prove the Max-Min optimality based on the difference equation in PA theory with a very clear and concise manner. The proofs of Ma and Cao (1994), Yao and Schechner (1989) are much more difficult respectively from the performance gradients and linear programming. The performance difference equation and policy iteration used in this paper give another way to do the optimization of queueing systems. The remainder of the paper is organized as follows. In Section 2 we give an introduction of PA theory in queueing systems related to this paper. Section 3 discusses the optimization problems in closed Jackson networks. We prove the Max-Min optimality of service rates based on the performance difference equation. The policy iteration algorithm is also introduced to do the optimization. In Section 4 we give the numerical experiments to demonstrate the optimality of Max-Min service rates. The difference between customer-average performance and time-average performance is also demonstrated by numerical examples. Finally, we conclude this paper in Section 5. 2 BACKGROUND ON PERTURBATION ANALYSIS In this section, we give an introduction of PA theory in queueing systems, especially the concept of perturbation realization factors and performance sensitivity equations in queueing systems. Consider a closed Jackson network consisting of M servers (Chen and Yao 21, Gordon and Newell 1967). The number of total customers in the network is N. After the service of a customer has been completed at server i, this customer will depart from server i and enter server j with routing probabilities q ij, i, j = 1,2,,M. Without loss of generality, we assume q ii =. The service discipline in each server is FCFS (First-Come First-Served) and the buffer size is infinite. Let n i be the number of customers at server i, and set n =(n 1,n 2,,n i,,n M ). The service requirement of each customer at every server is assumed to be exponentially distributed with mean one, and the service rate of server i depends on n and is therefore denoted as μ i,n, i = 1,2,,M. It is easy to see that n is the system state, and the state space is S = all n : M i=1 n i = N. We call this network a state-dependent closed Jackson network. Let n(t) denote the system state at time t and f (n) : S R =(, ) be the cost function. Let T L be the Lth service completion time of the network (counting all the service completions of servers in the network). The time-average performance is defined as η T = lim TL f (n(t))dt F L = lim, (1) T L T L where F L := T L f (n(t))dt, and the customer-average performance is defined as η ( f ) F L = lim L. (2) We assume that the state process n(t) is ergodic so that the limits in (1) and (2) exist. The time-average performance is a traditional metric used in many systems, while the customeraverage performance is also another important performance metric. These two performance metrics describe different aspects of the performance in queueing systems and both are widely used in practical applications. The difference between them will be discussed later. We now review the perturbation realization factors in PA theory. In a closed network, if the service completion time of a server is delayed by a small amount of time Δ, we say that the server has a perturbation. This perturbation will affect the system performance η ( f ). The effect of a single perturbation Δ of server k when the system is at state n can be measured by the perturbation realization factor, which is defined as c ( f ) ΔFL (n,k)= lim lim E Δ Δ =lim lim Δ E =lim lim Δ E F L F L Δ ( 1 T L Δ TL ) f (n (t))dt f (n(t))dt, (3) where n (t) is the state of the perturbed sample path at time t, T L is the Lth service completion time on the perturbed sample path. From (3), we define the perturbation realization probability c(n,k) as a special case of c ( f ) (n,k) where f (n) =I(n) 1 for all n S. c(n,k) is the probability with which a perturbation of server k at state n will be realized at every server of the network ultimately. With the perturbation realization factors c ( f ) (n,k), we may derive the system performance derivative with respect 51

3 to the service rates as follows (Cao 1994). dη ( f ) d = η(i) π(n)c ( f ) (n,k), (4) where η (I) is a special system performance corresponding to f (n)=i(n) 1 for all n S, and π(n) is the steady-state probability of state n. Since η (I) is a special case of η ( f ), from (2) we have η (I) = lim TL I(n(t))dt T L = lim L L. (5) It is easy to know that η (I) is the reciprocal of average throughput of the network, i.e., L η th = lim = 1, (6) T L η (I) where η th is denoted as the throughput of queueing networks. With (1), (2), and (5), we have η ( f ) F L T L = lim T L L = η T η (I), (7) which shows the relationship between the customer-average performance η ( f ) and the time-average performance η T. Since the service rates will affect the value of both η T and η (I), the optimal values of service rates for η ( f ) and η T are generally different. The numerical experiments in Section 4 demonstrate this point. If we substitute f (n)=i(n) into (4), we get the derivative equation of η (I) as below. dη (I) d = η(i) π(n)c(n,k). (8) These formulas (4) and (8) are important for the gradient-based algorithm of PA theory in queueing systems. We can estimate the performance derivative from sample paths and use the gradient-descent algorithms to optimize the system performance through adjusting the service rates (Cao 1994, Chong and Zak 21). 3 MAX-MIN OPTIMALITY OF SERVICE RATES In this section, we consider the optimization problem of a state-dependent closed Jackson network, where the optimization parameters are the service rates of each server k at each state n, k = 1,2,,M, n S. The objective is to minimize (or maximize) the customer-average performance η ( f ). With the notation of Markov decision processes (Puterman 1994), we can denote U =,k = 1,2,,M,n S as a stationary policy based on parameter. The total policy space is Ψ = all U, which is parameterized by. The cost function is f (n),n S, which may be different for different actions. Furthermore, we denote the cost function as f (n, μ n ), where μ n :=(μ 1,n, μ 2,n,,μ M,n ) is the action at state n. With this notation, a policy can be denoted as U = μ n,n S. With PA of queueing systems, we can use the performance derivative equation (4) to optimize the service rates. But this type of gradient-descent algorithms has a slow convergence speed and may be trapped in a local optimum. In the middle of 9 s of the last century, PA theory has been extended to Markov systems (Cao and Chen 1997, Cao 23). The performance difference equation and policy iteration algorithm are derived along this direction. But this methodology can only optimize the traditional timeaverage performance. Recently, the performance difference equation for customer-average performance is also derived (Xia, Chen, and Cao 28). It gives a new way to do optimization of queueing systems with customer-average performance criterion. With the relationship formula of perturbation realization factors between queueing systems and Markov systems (Xia and Cao 26b), researchers derive the customeraverage performance difference equation for queueing systems. Here we just list the main results without explanation. The detailed proof can be found in Xia, Chen, and Cao (28). When the service rates of server k at state n is changed to μ k,n, the customeraverage performance of closed Jackson networks will be changed from η ( f ) to η ( f ). The performance difference equation is derived as below. η ( f ) η ( f ) = η (I) π Δμk,n (n) c ( f ) (n,k)+h(n), (9) where Δ = μ k,n, h(n)= f (n) f (n), the superscript prime represents the parameters of the perturbed system. If the service rates of all the servers at all the states have changes, the performance difference equation is derived similarly as below. η ( f ) η ( f ) = η (I) π (n) n S M k=1 Δ c ( f ) (n,k)+h(n). (1) Furthermore, with the property of perturbation realization factors c ( f ) (n,k), equation (1) can be simplified as follows (Xia, Chen, and Cao 28). η ( f ) η ( f ) = η (I) π (n) n S f (n) c ( f ) (n,k). (11) M μ k,n k=1 511

4 With these performance difference equations, we can find the optimal service rates of closed Jackson networks have a Max-Min optimality if the cost functions satisfy some requirements. The detailed Max-Min optimality is described as below. Theorem 1 (Max-Min Optimality for State- Dependent Service Rates) In a state-dependent closed Jackson network, if f (n) =l (n)+ M k=1 l(n,k), l (n) and l(n, k) are constants, i.e., f (n) changes linearly with service rates, then the optimal service rate of each server can be either maximal or minimal. Proof. In a state-dependent closed Jackson network, when the service rate of server k changes from to +Δ, the cost function f at state n also changes from f (n) to f (n). Since f (n)=l (n)+ M k=1 l(n,k),wehaveh(n)= f (n) f (n)=δ l(n,k), where l(n,k) is a constant which means the changed cost function per unit changed service rate Δ. So, the performance difference equation (9) becomes η ( f ) η ( f ) = η (I) π Δμk,n (n) c ( f ) (n,k)+h(n) = η (I) π (n)δ c( f ) (n,k) + l(n,k). (12) With (12), we can prove Theorem 1 easily. We first assume that the optimal service rate is neither maximal nor minimal. Since η (I) and π (n) are always positive, from (12) it is easy to know that l(n,k) c( f ) (n,k) μ should be k,n zero (otherwise we can change to get a better η ( f ) ). Since l(n,k) c( f ) (n,k) is zero, it is obvious that the maximal or minimal value of can also achieve the optimal performance. Therefore, it is proved that we can get the optimal performance from the maximal or minimal values of service rates. This theorem extends the similar results of Ma and Cao (1994), Yao and Schechner (1989), which discuss the optimization of the time-average performance of load-dependent networks, to the optimization of the customer-average performance of state-dependent closed networks. It implies that in the performance optimization of closed queueing networks, if f changes linearly with service rates, we only need to consider the maximal or minimal service rates for every state n. So, the action space is reduced to 2 M. Using this property may speed up the optimization procedure greatly. Moreover, from ( 12) itis easy to know that η ( f ) is monotone with respect to if we only change and fix other parameters. So far, we have discussed the optimization of statedependent service rates in closed Jackson networks. The Max-Min optimality of service rates have been proved and it can be used to simplify the optimization procedure. With the performance difference equations ( 9) and (11), the policy iteration type of algorithms can be further derived. This type of algorithms can be implemented on-line based on the estimation of realization factors on a single sample path. The details of algorithm can be found in Xia, Chen, and Cao (28) and we will use it to do numerical experiments in Section 4. Below, we further discuss the optimization problem for loaddependent service rates in closed Jackson networks. We use the similar procedure to prove the Max-Min optimality for load-dependent service rates. In a load-dependent closed Jackson network, the service rate of a server k is dependent only on the queue length at server k. We denote the load-dependent service rates as k, k = 1,2,,M, n k =,1,,N. Obviously, it is known that k when n k =. As we know, the steady-state distribution of closed Jackson network has a product-form solution. With the product-form solution and customerbased aggregation of realization factors, the performance difference equation is derived as follows when the service rates are changed from k to μ k,n k for a particular server k and n k = 1,2,,N (Xia and Cao 26a). η ( f ) η ( f ) = η (I) N n k =1 π (n k ) Δμk,nk k c ( f ) (n k,k)+ h(n k,k), (13) where c ( f ) (n k,k) := π(n n k )c ( f ) (n,k), h(n k,k) := n S nk π(n n k )h(n,k)= π(n n k )[ f (n) f (n)], S nk is n S nk n S nk the set of states where the number of customers at server k is equal to n k. With the performance difference equation ( 13), we can also derive the Max-Min optimality of service rates in a loaddependent closed Jackson network. The proof is similar to that of Theorem 1 and it is neglected for the simplicity of paper. Theorem 2 (Max-Min Optimality for Load- Dependent Service Rates) In a load-dependent closed Jackson network, if f (n)=l (n)+ M k=1 l(n k,k)k,l (n) and l(n k,k) are constants, i.e., f (n) changes linearly with service rates k, then the optimal service rate of each server can be either maximal or minimal. With Theorem 2 we know if the cost function is linear with the service rates, we only need to choose the maximal or minimal service rates. It is similar to Theorem 1. These optimality properties can greatly simplify the optimization complexity of closed Jackson networks. With these theorems, we can only focus on the maximal or minimal values of service rates when we apply any optimization algorithm. This is also called the Bang-Bang control in the area of control science. 512

5 Table 1: Optimal service rates of a state-dependent closed Jackson network. n μ1,n μ2,n μ3,n D μ1,n D μ2,n D μ3,n μ1,n μ2,n μ3,n μ 1,n μ 2,n μ 3,n (,,5) [.1,2.] [.3,1.5] [.9,3.] (,1,4) [.2,2.5] [.2,1.5] [.3,3.5] (,2,3) [.1,1.5] [.5,2.5] [.5,4.5] (,3,2) [.3,1.5] [.6,3.] [.1,1.5] (,4,1) [.2,2.] [.9,2.] [.4,2.] (,5,) [.5,1.5] [.4,2.5] [.1,3.] (1,,4) [.4,2.] [.5,3.] [.6,2.] (1,1,3) [.5,2.5] [.2,3.5] [.8,1.5] (1,2,2) [.2,2.] [.6,1.5] [.4,2.5] (1,3,1) [.6,1.5] [.3,2.] [.2,2.5] (1,4,) [.4,2.5] [.5,3.5] [.7,3.5] (2,,3) [.6,1.] [.3,3.5] [.1,3.] (2,1,2) [.7,1.5] [.4,4.] [.6,2.] (2,2,1) [.8,2.5] [.1,2.] [.5,3.5] (2,3,) [.7,2.] [.2,1.5] [.1,4.] (3,,2) [.3,2.] [.1,1.] [.4,2.] (3,1,1) [.5,2.5] [.1,2.] [.8,4.] (3,2,) [.9,4.5] [.6,3.5] [.6,3.5] (4,,1) [.1,3.5] [.8,1.5] [.9,2.] (4,1,) [.6,4.] [.4,3.] [.4,2.5] (5,,) [.4,4.5] [.6,3.5] [.5,2.] NUMERICAL EXPERIMENTS In this section, we use the policy iteration algorithm to optimize the service rates of closed Jackson networks. The numerical results demonstrate the Max-Min optimality of service rates. The detailed implementation of policy iteration algorithm is neglected for the length limitation of paper. Readers who are interested can refer the literature Xia, Chen, and Cao (28) and Xia and Cao (26a), which are respectively for statedependent and load-dependent closed Jackson networks. First, we consider the optimization of state-dependent service rates in closed Jackson network. The number of servers is M = 3 and the number of customers is N = 5. The routing probability matrix is Q = The initial service rates μk,n are listed in Table 1. The cost function is defined as f (n)=n 1 + r 3 k=1, where r is the cost of unit service rate and we set r = 1 in this example. In fact, r can be a set of values which represent the different costs at different states. The objective is to minimize the customer-average performance η ( f ) through adjusting the service rates, k = 1,2,3 and n S. The value domains of service rates are listed as D μk,n in Table 1. With the policy iteration algorithm for customer-average performance (Xia, Chen, and Cao 28), we find the algorithm iterates some times and locates the optimal solution finally. The optimal service rates μk,n are listed in Table 1. The minimal customer-average performance is η ( f ) = From Table 1 we can also see that the optimal service rates are either maximal or minimal. It demonstrates the correctness of Theorem 1. As a comparison, we use the traditional algorithms in Markov decision processes to optimize the time-average performance of this problem. With this performance metric, we obtain the optimal service rates μ k,n for time-average performance, which are listed in Table 1. The corresponding η ( f ) = , which is larger than the optimal value η ( f ) = The optimal service rates μ k,n for time-average performance are also different from the optimal service rates μk,n for customer-average performance. This experiment illustrates that the algorithm for time-average performance can not be directly used to optimize the customer-average performance. These two performance metrics characterize the different aspects of optimization problems. 513

6 Furthermore, in order to more clearly illustrate the difference between customer-average performance and timeaverage performance, we give another numerical example as follows. Consider a cyclic network with 2 servers. The number of customers is N = 3. For simplicity, we can use n 1 to represent the system state n. The service rates of server 2isfixedatμ 2,n1 = 1, n 1 =,1,2,3. The service rates of server 1 is μ 1,n1 = 1 when n 1 =,1,2, and μ 1,n1 [.5,2] when n 1 = 3. The cost function is f ()= f (1)= f (2)=1, f (3) = μ 1,3 (1 μ 1,3 ) 2. The objective is to choose an optimal μ 1,3 to minimize the system performance η ( f ). With numerical computation, we obtain the performance curve as Figure 1. It is obvious that the optimal μ 1,3 is 2 for η ( f ), while the optimal μ 1,3 is.5 for η T. This example clearly demonstrates the difference between these two performance metrics. system performance η T η (f) μ 1,3 Figure 1: Difference between customer-average performance and time-average performance. The above numerical experiment is about the statedependent service rates. The situation for load-dependent service rates is similar and here is not presented for the paper concision. Interested readers can refer the numerical experiment by Xia and Cao (26a), which can also demonstrate the Max-Min optimality of load-dependent service rates. 5 CONCLUSION In this paper we discuss the service rates optimization problem in closed Jackson networks. With the difference equation of customer-average performance, we prove that the optimal service rates have a so-called Max-Min optimality. This optimality is valid for both state-dependent and load-dependent closed Jackson networks. With the Max-Min optimality, we only need to choose the maximal or minimal values for service rates in optimization procedure. It simplifies greatly the complexity of such type of optimization problems. ACKNOWLEDGMENTS The first author would like to thank Prof. Xi-Ren Cao for his instruction and help when the author did this research work in his Ph.D. pursuing period. Of course, the possible errors in this paper are solely the responsibility of the paper authors. REFERENCES Cao, X. R Realization probabilities - the dynamics of queueing systems. New York: Springer Verlag. Cao, X. R. 23. From perturbation analysis to markov decision processes and reinforcement learning. Discrete Event Dynamic Systems: Theory and Applications 13:9 39. Cao, X. R., and H. F. Chen Potentials, perturbation realization, and sensitivity analysis of markov processes. IEEE Transactions on Automatic Control 42: Cassandras, C., and S. Lafortune Introduction to discrete event systems. Boston, MA: Kluwer Academic Publishers. Chen, H., and D. Yao. 21. Fundamentals of queueing networks: Performance, asymptotics, and optimization. New York: Springer Verlag. Chong, E. K. P., and S. H. Zak. 21. An introduction to optimization, 2 nd edition. New York: John Wiley & Sons. Glasserman, P Gradient estimation via perturbation analysis. Norwell, MA: Kluwer Academic Publisher. Gong, W. B., and Y. C. Ho Smoothed (conditional) perturbation analysis of discrete event dynamical systems. IEEE Transactions on Automatic Control 32: Gordon, W. J., and G. F. Newell Closed queueing systems with exponential servers. Operations Research 15: Ho, Y. C., and X. R. Cao Perturbation analysis of discrete-event dynamic systems. Boston, MA: Kluwer Academic Publisher. Ho, Y. C., X. R. Cao, and C. Cassandras Infinitesimal and finite perturbation analysis for queueing networks. Automatica 19: Ma, D. J., and X. R. Cao A direct approach to decentralized control of service rates in a closed jackson network. IEEE Transactions on Automatic Control 39: Puterman, M. L Markov decision processes: Discrete stochastic dynamic programming. New York: John Wiley & Sons. 514

7 Xia, L., and X. R. Cao. 26a. Aggregation of perturbation realization factors and service rate-based policy iteration for queueing systems. In Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, CA, USA, Xia, L., and X. R. Cao. 26b. Relationship between perturbation realization factors with queueing models and markov models. IEEE Transactions on Automatic Control 51: Xia, L., X. Chen, and X. R. Cao. 28. Policy iteration of customer-average performance in queueing systems. Submitted to Automatica. Yao, D. D., and Z. Schechner Decentralized control of service rates in a closed jackson network. IEEE Transactions on Automatic Control 34: AUTHOR BIOGRAPHIES LI XIA received the B.E. degree in automation in July 22 and the Ph.D. degree in control science and engineering in July 27, both from Tsinghua University, Beijing, China. He is currently a research staff at IBM China Research Laboratory, Beijing, China. His research interests include discrete event dynamic systems (DEDS) theory and applications, simulation optimization techniques, and supply chain management. His address is <crlxiali@cn.ibm.com>. MING XIE received his Ph.D. degree in Tsinghua University from China in 26 and then joined IBM China Research Laboratory. Currently, his research interests include supply chain management, agent-based simulation, complex systems and business intelligence. His address is <xieming@cn.ibm.com>. WENJUN YIN received his Ph.D. degree in Tsinghua University from China in 24 and then joined IBM China Research Laboratory. Currently, his research interests include supply chain management and logistics, business analytics and optimization, data mining and business intelligence. His address is <yinwenj@cn.ibm.com>. JIN DONG, Manager of Supply Chain Management and Logistics Research in IBM China Research Laboratory. He received his Ph.D. degree in Tsinghua University from China in 21. Before joined IBM, he was the Research Assistant Professor in Industrial Engineering Department of Arizona State University in USA. His address is <dongjin@cn.ibm.com>. Xia et al. 515