Considerations on Distributed Load Balancing for Fully Heterogeneous Machines: Two Particular Cases

Transcription

1 Considerations on Distributed Load Balancing for Fully Heterogeneous Machines: Two Particular Cases Nathanaël Cheriere Departent of Coputer Science ENS Rennes Rennes, France Erik Saule Departent of Coputer Science University of North Carolina at Charlotte Charlotte, USA Abstract When the size of parallel systes increases, centralized algoriths to schedule tasks on the syste can induce a significant overhead. This is why decentralized scheduling algoriths have been developed. The ost popular one certainly is work-stealing because of its interesting theoretical guarantees. Parallel systes have evolved fro hoogeneous clusters to fully heterogeneous ones such as GPU-accelerated clusters. We investigate in this paper decentralized scheduling algoriths for heterogeneous systes. The guarantees of workstealing algoriths no longer hold on such systes because it is an a posteriori algorith which highly depends on the initial distribution of work. We focus on a priori decentralized scheduling algoriths for heterogeneous systes and we propose two distributed algoriths to balance the load on unrelated achines for two particular cases. The first one exploits a low heterogeneity in the task set and reaches an approxiation ratio linear in the nuber of types of tasks. The second one focuses on the case where the syste only uses two different types of achines and we show it is a 2-approxiation if the syste converges. In the case it does not converge, we study the dynaic equilibriu of the syste. In the hoogeneous case, we nuerically copute the probability density function of the load ibalance and show that the ibalance is low on average. And we show using siulation that the heterogeneous case is siilar to the hoogeneous case and that the ibalance is low in both cases. Keywords-Unrelated Machine Scheduling; Load-Balancing; Decentralized Algoriths; Approxiation Algoriths I. INTRODUCTION Scheduling the execution of a parallel progra on ultiple achines is one of the basic probles in parallel coputing. Its solution is far fro obvious as any hypotheses can be done to specify the conditions, especially on the heterogeneity of the achines used to execute the progra. These probles have been studied since the 6s, but often fro a centralized point of view. However, with the current size of parallel systes, the cost of the resolution of the proble on one achine cannot be overlooked anyore [16]. Centralized systes have a high overhead when the nuber of anaged processors increases. Even a siple least loaded achine first scheduling policy takes a linear tie in the nuber of tasks and logarithic in the nuber of achines [12]. This obviously does not scale well when the size of the syste increases and does not handle at all the inherent iprecision of all scheduling systes (runties are tycally difficult to predict). Decentralized load balancing algoriths have been designed to alleviate the bottleneck of having a single processor doing all the scheduling. The work stealing strategy, introduced in [7] and odified for Cilk [6], is certainly the ost popular decentralized scheduler: every achine is responsible for its own charge, and when a achine no longer has jobs to execute, it tries to steal soe fro another achine. This strategy has guaranteed perforance 1 when the achines are identical [6] and even when the achines process the tasks at different speeds [1]. However odern achines are coposed of fully heterogeneous processors. Two processors, such as a CPU and a GPU, are not just different in ter of their speed, they also differ in capabilities: for soe tasks a CPU ight be fast and a GPU slow while it could be the reverse for other tasks. Moreover, in the case of two CPU with different architecture, a task ight be optiized for a specific architecture and perfor worse on the other architecture. Coon decentralized load balancing strategies could be applied to these systes. But they would no longer guarantee anything on the quality of the derived solution. In this work, we investigate the proble of decentralized load-balancing for heterogeneous achines under the perspective of provable perforance. The contribution of this work are as follow: We show in Theore 1 that work stealing policy can lead to arbitrarily bad load balance in the fully heterogeneous case. We provide algorith MJTB which is decentralized and converges to a solution no worse than k the optial solution assuing there are only k groups of tasks (Theore 5). 1 Guaranteed refers to an approxiation ratio; that is to say theoretically proven upper bound of the ratio between what the algorith obtains and the optial solution. Though in this case the guarantee is not achieved in the worst case but in average.

2 We provide an iterative decentralized algorith, DLB2C, for systes with only two types of processors (such as a CPU-GPU cluster) and show that if the algorith converges, then the obtained solution is a 2- approxiation (Theore 7) In cases where DLB2C does not converge to a single solution, we study its behavior by odeling the one cluster case using arkov chains and show the ibalance is bounded with high probability (Figure 2). We also show using siulations that the two clusters case behaves like the one cluster case (Figure 4) and that a solution no worse than 1.5 ties a guaranteed centralized solution is tycally reached within a few iterations (Figure 5). The docuent is organized as follows. We forally introduce the proble in Section II. We quickly survey results related to our proble in Section III. Section IV discusses a posteriori and a priori load balancing algoriths. Then we investigate two cases for a priori algoriths. We investigate instances where the nuber of types of tasks is sall in Section V. Section VI presents a decentralized algorith for the case where there are only two types of processors and we show that if it converges the obtained solution is a 2-approxiation. We study the case where it does not converge using arkov odels and siulation in Section VII. Section VIII concludes this docuent. II. PROBLEM DEFINITION The proble studied in this work is a load balancing proble, the goal is to construct a partition S of a set of sequential independent jobs J onto the achines in the set M in order to iniize an objective function. A job is a process that can only be executed on one achine at a tie and it is independent fro other jobs. We also assue that it has to be executed without interruptions to be copleted. The achines do not share eory. For the purpose of this paper we will use interchangeably the words task and job. Also, we will use interchangeably the words processor and achine. Many functions can be iniized but we focus on the akespan which is the tie at which all the jobs have been executed. We denote by p i,j the tie needed for the execution of job j on the achine i. And S(i) is the set of jobs assigned to achine i. We denote the akespan as C ax (S) = ax i M C(S, i) where C(S, i) = j S(i) p i,j is the tie when all the jobs of achine i have been copleted. In the following parts, we will denote C(S, i) as C(i) when there is no abiguity on the solution used to copute the akespan. In this proble, the partition obtained is usually copared to the iniu possible akespan OPT which characterizes the optial solutions, OPT = in S C ax (S). The achines can be hoogeneous, heterogeneous related, or heterogeneous unrelated. The achine set can be hoogeneous (identical). In this case, the achines are said to be unifor, each job j can be executed on every achine i within the sae aount of tie, i, i M, p i,j = p i,j. In the heterogeneous related case, all the achines are different but only differ by a fixed factor. For all the jobs we have i, i M, α, j J, p i,j = αp i,j. The heterogeneous unrelated (fully heterogeneous) case is the ost general. There is no assuption on the relation of processing ties of the tasks on the processors, p i,j is an arbitrary value (which can be infinite). The centralized version of the proble we are interested in is known as scheduling independent tasks on unrelated parallel achines to iniize the akespan and is denoted R C ax in the classical 3 field notation of Graha et al. [13]. This proble is known to be NP-Coplete [1]. Here we are interested in the decentralized version of this proble. Therefore we will assue that the jobs have an arbitrary initial distribution. This could apply in cases where there is a static set of tasks which is pre-distributed. But it could also apply in cases where tasks directly appear on processors. Coon reasons for this are if a task can locally spawn ore tasks or if tasks are subitted to a particular processor. III. RELATED WORK Certainly the ost faous algorith for these kinds of proble is List Scheduling proposed by Graha in 1966 [11] which consists in scheduling tasks greedily on the first achine available. On identical achines, this algorith is a 2-approxiation algorith: the generated solution S is such that C ax (S) 2 OPT. In the offline case, the tasks are known in advance and one can order the tasks using a Largest Processing Task (LPT) first order, and then List Scheduling becoes a 4 3 -approxiation algorith [12]. Later, dual approxiation ethods lead the way to Polynoial Tie Approxiation Schees (PTAS) for the identical achine case; they have a higher coplexity but their approxiation ratio can be arbitrarily close to 1 [15]. The proble of scheduling tasks on fully heterogeneous achines is not as well solved. Lawler and Labetoulle showed that the proble with pre-eption 2 can be solved in polynoial tie using linear prograing [18]. The proble without pre-eption can be approxiated within a factor 2 also using a linear prograing proble but then using intelligent rounding techniques to reconstruct an integer solution [2]. A faster algorith using unsplittable flow has been designed, but still is a 2-approxiation 2 The possibility to pause a job on a achine and restart later it on another achine

3 Algorith 1: Work Stealing for a particular achine Data: achine Data: S() jobs assigned to while true do if S() = then Select randoly a target achine i if S(i) then Steal half of the non executed jobs of i Start running a job j of S() Reove j of S() algorith [9]. It was also shown that the proble cannot be approxiated with a better approxiation ratio than 3 2 unless P = NP [2]. However, if the nuber of processors is fixed then approxiation schees can be designed (for instance if there are only two processors [17]). All the previously entioned works propose offline centralized algoriths. A coon approach for the identical achine case is to ake the syste online by scheduling the tasks at the tie of their subission. Tycally such systes rely on the greedy properties of List Scheduling and aintain the load of each achine in a priority queue to schedule each task on the least loaded processor in O(log M ) and guarantees that each interediate solution is a 2-approxiation [11]. This does not scale with the nuber of processors and is inherently centralized. Approaches based on the balls in bins proble trade accuracy for coplexity. Instead of cking the least loaded processor, the algorith cks the least loaded processor of d randoly (uniforly) chosen processors in O(d) and akes an error of roughly ln ln J / ln d [4]. This principle has even been successfully adapted in the heterogeneous related case [2], [3]. These ethods are fully decentralized if one assues that the load of a processor can be probed externally but does not apply to fully heterogeneous systes. In the context of heterogeneous systes, Chen et al. addressed the proble of load balancing at subission tie for clusters coposed of two sets of identical achines. Machines within the sae set are identical but two achines fro different sets can be heterogeneous unrelated. They proposed a centralized 4-approxiation algorith [8]. The ost coon decentralized scheduler that does not distribute the tasks at subission tie is the work-stealing strategy (Algorith 1) introduced by Burton and Sleep [7]. This strategy is essentially a decentralized ipleentation of List Scheduling [21]. Each achine aintains a list of locally available tasks and when a achine finishes its last local job, it contacts its neighbours and tries to steal soe of the non running jobs. This idea was applied to the Cilk iddleware and soe guarantees on the execution tie of a batch of jobs using a work stealing algorith on identical parallel achines have been derived [5], [6]. In particular, the expected axiu akespan is bounded by the average work per achine plus a big-o of the critical path 3 p of the proble, E(C ax (S)) j J p i,j/ M +O(p ). This result was later extended to the proble with heterogeneous related achines and the possibility to use pre-eption [1]. IV. A PRIORI LOAD BALANCING Load balancing at the task s subission tie works in the case of identical achines (or eventually related achines) essentially because List Scheduling is a good greedy strategy for this proble. But there are no known good greedy algoriths for the heterogeneous case; which is why algoriths that work in this case only work under particular hypotheses, such as liiting the nuber of types of processors to two [8]. The inner difficulty of the proble akes the finding of a generally applicable ethod unlikely. On the other hand, applying a work stealing strategy on unrelated achines has a shortcoing. Indeed, this a posteriori strategy starts stealing jobs fro another achine only when the work previously scheduled on the achine has been executed. But if the initial distribution is poorly done (like in Table I), the first steal can happen long after the optial akespan. Theore 1. Applying a work stealing strategy on unrelated achines can induce an unbounded akespan. Proof: Table I presents the p i,j values of a 5 tasks, 3 achines instance of the proble. If the initial distribution follows the allocation denoted by a circle in the table, the first steal operation only happens after a tie n, and the execution can be finished in n + 1 units of tie. However, with a good schedule the work can be finished in 2 units of tie. Cost Machine A Machine B Machine C Job 1 1 n n Job n Job 3 n 1 1 Job 4 n 1 1 Job 5 n 1 1 Table I THE CIRCLED DISTRIBUTION IS A BAD INITIAL DISTRIBUTION TO APPLY A Work Stealing STRATEGY; THE FIRST STEAL CAN ONLY HAPPEN AFTER n UNITS OF TIME. One cannot rely on an efficient distribution of the tasks at subission tie. And one cannot wait for a achine to finish processing its tasks to rebalance the work. The only reaining option would be a priori load balancing: balancing the load before processing the tasks. To be decentralized, we choose to investigate algoriths that balance optially (or at least in a guaranteed way) the load of a subset of the processors. Obviously cking a large subset of the processors is essentially a centralized solution. 3 The algorith is defined on graph of dependent task. The critical path denotes the length of the longest sequence of operations that ust be sequential. In the independent task odel, the processing tie of the longest task.

4 Therefore we focus on algoriths that balance the work by pairs. The following theore infors us that such an approach ight have a poor guarantee in soe cases. Proposition 2. A generic algorith balancing optially each pair of achine can induce an unbounded akespan. Proof: It is possible to construct a situation with 3 achines and 3 jobs where all pairs of achines have their load optially distributed but the global load is not optially distributed and can even be infinitely longer than the optial one. This situation is described in Figure II. Each job can be run fast, slow or very slow. When a pair of achines rebalances their load, there are two jobs, one of which can be run fast and one that can not. If the job that can be run fast is scheduled on its fast processor, then the reaining task ust either be executed at very slow rate, or put two tasks on the sae achine: one slow and one fast. Both solutions have a worst akespan than the original solution which copletes in n tie units. It is clear that the optial schedule has a akespan of 1, but the algorith is stuck in a solution of akespan n. Cost Machine A Machine B Machine C Job 1 1 n n 2 Job 2 n 2 1 n Job 3 n n 2 1 Table II A SITUATION WITH 3 MACHINES AND 3 JOBS WHERE PAIR-WISE BALANCING LEADS TO AN UNBOUNDED MAKESPAN. THE OPTIMAL MAKESPAN IN THIS SITUATION IS 1 WITH JOB 1 ON MACHINE A, JOB 2 ON MACHINE B AND JOB 3 ON MACHINE C. HOWEVER WITH THE CIRCLED DISTRIBUTION, THE SCHEDULE OF ALL PAIRS OF MACHINES IS OPTIMAL, DESPITE THE MAKESPAN IS n. This proposition indicates that there is little hope to find generic ethods with constant approxiation ratio, so we look at particular cases. We would like to share a few rearks on the use and applicability of a priori algoriths before studying the further. First, we say the balancing is done before the execution of the tasks for siplicity. But it could be done after each task subission to address the lack of good greedy property or siply done periodically. By running it periodically, an a priori load balancer can naturally take into account the dynaicity of the coputing syste, soe tasks ight be shorter or longer than predicted, and soe tasks ight dynaically be created on a processor. These scenarios are easily taken into account contrarily to odels that balance tasks at subission tie only. Also, we will see later that these load balancers ight not converge to a unique solution, therefore it ight be necessary to run the balancing algorith concurrently with the application. A coon concern is that the algorith ight ove tasks any tie causing the data of the tasks to be continously bounced around the network. We showed previously in a workstealing based load balancer [14] that it is also possible to separate, within a runtie syste, the load balancing itself and the data oveent. This allows to optiize the data oveents and to iniize the ipact of oving the data any ties in the coputing syste. V. LOAD BALANCING PER TYPE OF JOB In this section, the jobs are grouped by type. In each type, all the jobs have the sae processing tie: j, j T, j has the sae type as j i M, p i,j = p i,j. This distinction can easily be ade in real systes where siple queries can represent ost of the jobs of a syste: even if the jobs are not exactly the sae, their processing tie is siilar and only vary depending on which achine executes the. We first study the case where there is only one type of job and provide an optial decentralized algorith, and then extend the algorith to derive a guaranteed decentralized algorith in the general case. A. Balancing only one type of job Algorith 2: Basic Greedy Data: achines and i Data: S, a distribution of jobs A := S(i) S() S() := S(i) := while A do let j be a job in A if C() + p,j C(i) + p i,j then S() := S() {j} S(i) := S(i) {j} A := A\{j} Algorith 3: OJTB Data:, the host achine Data: S, an initial distribution of the jobs while true do Select randoly (uniforly) i M Distribute optially the load of i and with the Basic Greedy The One Job Type Balancing algorith, denoted as OJTB (Algorith 3) is run by each achine. It is quite siple. It randoly chooses a target and balances the load of the two achines in an optial way. Getting the optial load balancing for two achines in this proble is ensured by Basic Greedy (Algorith 2) thanks to the fact that the jobs

5 cost is defined by the achine only. We prove that OJTB is optial. Lea 3. Basic Greedy provides an optial distribution when there is only one type of jobs. Proof: Since there are only two achines and all the jobs are equivalent, the optial solution is obviously given by a greedy algorith using Earliest Copletion Tie. Lea 4. OJTB (Algorith 3) converges to a distribution S of the jobs which is optial. Proof: Let S(n) be the solution created after the n-th execution of the algorith. Note that C ax (S(n + 1)) C ax (S(n)); If not, the balancing done between the two achines at step n + 1 would not be optial. Hence the function C ax (S(n)) is non-increasing. Let S be the distribution created by Algorith 3 after an infinite nuber of executions. As all the jobs have the sae processing tie we denote as p i the processing tie of any job on achine i. We can now represent S(i) only by its cardinality which we denote as N(i), N(i) = S(i). We also have C(i) = N(i)p i. Let S be an optial distribution and N (i) = S (i). By contradiction, let us assue that C ax (S) > C ax (S ). There exists i ax M such that C(i ax ) = C ax (S). In particular, N(i ax ) > N (i ax ), but this also iplies that there exists i such that N(i) < N (i) (because i M N(i) = J = i M N (i)). So at least one job can be oved fro i ax to i to have a better local distribution (C ax would decrease). Hence the distribution S is not optial for the pair of achine i ax and i but as Algorith 3 has been applied an infinite nuber of ties and because C ax is non-increasing, S should be optial for i ax and i. We have C ax (S) C ax (S ), hence C ax (S) = C ax (S ). B. Extension to ultiple types of jobs OJTB can be extended into the Multiple Job Type Balancing (MJTB) algorith to balance ultiple types of jobs; however, the perforance guarantee of the algorith becoes linear in the nuber of types of jobs (denoted as k in this section). Theore 5. MJTB (Algorith 4) applied to k types of jobs, converges to a distribution S of the jobs which is a k-approxiation. Proof: Let T l be the set of jobs of type l, if j and j are in T l, then i M, p i,j = p i,j. Moreover l, l such that l l, j T l and j T l, i M, p i,j p i,j. Let k be the nuber of distinct types of jobs. Algorith 3 is applied for all the types. Type l is optially distributed so the akespan C(T l ) of the distribution of T l Algorith 4: MJTB Data:, the host achine Data: S, initial distribution of the jobs Data: k, nuber of types of jobs Result: C ax (S) k OPT while true do Select i M foreach l: type of job do Distribute the jobs of type l between i and using Basic Greedy is saller than OPT. When all the types of jobs have been partitioned, we have C ax k i=1 C(T i) kopt. VI. LOAD BALANCING WITH TWO TYPES OF MACHINES In this section, we liit the proble to two different clusters of identical achines M 1 and M 2 ( i, i M 1 (resp. M 2 ), j J, p i,j = p i,j). This is eaningful in practice since the advent of GPU-accelerated clusters. We develop a new algorith with a proven approxiation ratio of 2 under the assuption that the cost of any task is saller than the optial akespan for the proble, which is forally expressed as i M, j J, p i,j OPT. This hypothesis is realistic in the sense that there is tycally no wild variation in what heterogeneous achines can do. For instance, GPUs used to be thought as a hundred tie faster than CPUs. However, careful analysis show that the ratio is uch saller [19]. It also supposes that there is not a job which can only run on one cluster (and if this was the case, they could be assigned beforehand). Moreover, this hypothesis also suggests that there is a large aount of work, which is the starting point for creating a decentralized algorith. To siplify the notations, in this section we will denote as p i,j the cost of job j on cluster i. We first focus on the centralized version of the proble and then use this centralized algorith as a stepng stone to create a decentralized one. A. A centralized algorith This proble is a sub-proble of the scheduling proble of unrelated achines which has been studied by Lenstra et al. [2]. They provide an algorith with an approxiation ratio of 2. However, the algorith requires solving a linear progra which sees difficult to decentralize reasonably. This is why we developed a new greedy algorith to balance the load between two clusters. The idea behind this new algorith, called CLB2C (Algorith 5) for Centralized Load Balancing for Two Clusters, is quite siple: we first sort the jobs according to the ratio p 1,j /p 2,j so that the jobs which are executed faster on the first cluster are at the beginning of the list and the one executed faster on the second cluster are at the end of the

6 list. Then we evaluate the decision of placing the first job of the list on the achine of the first cluster with inial akespan and placing the last job of the list on the second cluster on the achine with inial akespan. The job placed is the one that iniizes the akespan of those two achines. That way, if a job is not placed on the cluster where it can be executed at its inial cost, we know this choice does not have a significant ipact on the overall akespan. Algorith 5: CLB2C Data: M 1, The cluster of identical achines of type 1 Data: M 2, The cluster of identical achines of type 2 Data: J, The list of n jobs such that j {1,..., n}, p 1,J(j) OPT and p 2,J(j) OPT Result: A partition S of the jobs for each achine Sort jobs in J in increasing order of p 1,j /p 2,j (J(j) denote the j th job in J) Initialize S with one epty set per achine j 1 := 1 j 2 := n while j 1 j 2 do Select i 1 M 1 such that C(i 1 ) = in i M 1 C(i) Select i 2 M 2 such that C(i 2 ) = in i M 2 C(i) if C(i 1 ) + p 1,J(j 1 ) C(i 2 ) + p 2,J(j 2 ) then S(i 1 ) := S(i 1 ) {J(j 1 )} j 1 := j S(i 2 ) := S(i 2 ) {J(j 2 )} j 2 := j 2 1 return S Theore 6. CLB2C (Algorith 5) is a 2-approxiation algorith. C ax (S) 2OPT. Proof: Let i ax be a achine of M = M 1 M 2 such that C ax (S) = C(i ax ). We can suppose, without loss of generality, that i ax M 1. Let j ax be the last job placed on i ax. Let S be the incoplete partition of J just before the choice is ade by Algorith 5 to place j ax on i ax. Let C 1 be C(S, i ax ) and C 2 be such that C 2 = in i M 2 C(S, i). Note that C 1 has also the property C 1 = in i M 1 C(S, i). Indeed, the next job placed is j ax, which is placed on i ax, so C 1 = in i M 1 C(S, i). Let us copare in(c 1, C 2 ) to OPT. Let S 1 be the jobs placed on cluster 1 in S, and S 2 the ones placed on cluster 2, S k = i M k S (i). To copare C 1 and C 2, we will use the notion of work, which is defined as the processing tie of the jobs assigned to the achines. The work done on the cluster 1, W 1, has the property W 1 M 1 C 1, siilarly, the work W 2 done on cluster 2 is such that W 2 M 2 C 2. Let S be an optial solution with S 1 the set of jobs placed on cluster 1 and S 2 the set of jobs placed on cluster 2. The work done on cluster 1 with an optial solution is denoted as W 1, the work done on cluster 2 is denoted as W 2 To create an optial solution fro S, the jobs J 1 = S 1 S 2 should be oved fro the cluster 1 to the cluster 2, and the jobs J 2 = S 2 S 1 fro cluster 2 to cluster 1. We have the equations W 1 = W 1 j J p 1 1,j + j J p 2 1,j and W 2 = W 2 j J p 2 2,j + j J p 2,j. 1 By contradiction, let us assue that W 1 < W 1 and W 2 < W 2, fro the two previous equations we deduce p 1,j < p 1,j and p 2,j < p 2,j (1) j J 2 j J 1 j J 1 j J 2 Let α = p 1,jax /p 2,jax. Because the algorith sorts the jobs, we have j J 1, p 1,j /p 2,j α and j J 2, p 1,j /p 2,j α and with Equation 1 we have p 1,j < p 1,j α p 2,j < α p 2,j (2) j J 2 j J 1 j J 1 j J 2 But we also have j J 2 p 1,j α j J 2 p 2,j, which is in contradiction with Equation 2. Fro this, we deduce that W 1 W 1 or W 2 W 2, in particular, OPT ax(w 1 / M 1, W 2 / M 2 ) in(w 1 / M 1, W 2 / M 2 ) in(c 1, C 2 ). In conclusion, in(c 1, C 2 ) OPT. Let j be the job copared to j ax when j ax is placed. Because j ax has been placed on i ax fro the first cluster, we have C 1 + p 1,jax C 2 + p 2,j. If C 1 C 2, C 1 OPT and p 1,jax OPT so C ax (S) 2OPT. Else C 2 < C 1, so we have C 2 OPT, p 2,j OPT and C 1 +p 1,jax C 2 +p 2,j so C ax (S) 2OPT. In both cases, C ax (S) 2OPT. B. Decentralized algorith CLB2C is the base for a decentralized algorith to balance the load of achines on two clusters of unifor achines. Indeed, like the peer to peer approach fro the Work Stealing algorith the algorith DLB2C (for Decentralized Load Balancing for Two Clusters, Algorith 7) is executed on each achine: each achine randoly selects a target (a achine), and if both achines are fro the sae cluster, a Greedy Load Balancing is applied, if both achines are fro different clusters, CLB2C (Algorith 5) is used to balance both achines (considering two sub-clusters of one achine each). This algorith uses Greedy Load Balancing to balance two achines fro the sae cluster (Algorith 6). DLB2C ensures an interesting property: when the situation is stable, the akespan of the job distribution is bounded by twice the optial akespan.

7 Algorith 6: Greedy Load Balancing Data: 1, 2 achines of the sae cluster to balance Data: S distribution of jobs A = S( 1 ) S( 2 ) if 1 is in cluster 1 then Sort jobs in A in increasing order of p 1,j /p 2,j Sort jobs in A in increasing order of p 2,j /p 1,j Initialize S with one epty set for 1 and 2 while A do j first job in A if C( 1 ) C( 2 ) then S( 1 ) := S( 1 ) {j} S( 2 ) := S( 2 ) {j} A := A\{j} return S Algorith 7: DLB2C Data:, the host achine Data: J() Set of jobs initially on Result: C(J()) 2 OPT while true do Select i M if i is in the sae cluster as then Apply Greedy Load Balancing to i and M 1 := {} M 2 := {i} Apply CLB2C to M 1 and M 2 with the set of jobs J() J(i) Fro these we conclude that j 1 S 1, j 2 S 2, p 1,j 1/p 2,j 1 p 1,j 2/p 2,j 2 (3) Let i ax be a achine such that C ax (S) = C(i ax ). Without any loss of generality, we assue that i ax M 1. Let j ax be a job assigned to i ax such that p 1,jax /p 2,jax = ax j Siax p 1,j /p 2,j. Let C 1 = C(i ax ) p 1,jax. Since Greedy Load Balancing has been applied between the achines of the sae cluster, we have the property i M 1, C 1 C(i) (4) Let C 2 and i 2 be such that C 2 = C(i 2 ) = in i M 2 C(i). Using the sae reasoning by contradiction as in the proof of Theore 6 with α = p 1,j 1/p 2,j 1, we get in(c 1, C 2 ) OPT. Let us consider an execution of CLB2C between i ax and i 2. if j ax is not the last job placed by the algorith, then j ax has been copared to j which has been placed afterward, so C 1 +p 1,jax C 2 but C 1 +p 1,jax C 2 (because C 1 + p 1,jax = C ax (S)). Hence C 1 C 2 so we deduce C 1 OPT and with p 1,jax OPT we have C ax (S) 2OPT if j ax is the last job placed by the algorith, we have C 1 + p 1,jax C 2 + p 2,jax. Hence C ax (S) 2OPT. In both cases we have C ax (S) 2OPT. VII. DYNAMIC EQUILIBRIUM OF DLB2C Theore 7. If the distribution S provided by the execution of Algorith 7 becoes stable (for every pair of achine, the algorith does not ove any job), S is such that C ax (S) 2OPT. Proof: Let S 1 be the set of jobs assigned to cluster M 1 (S 1 = M S()) and 1 S2 the set of jobs assigned to cluster M 2. We first show that the jobs in S 2 have a ratio p 1,j /p 2,j larger the ones of the jobs in S 1. Let j 1 be such that p 1,j 1/p 2,j 1 = ax j S 1 p 1,j /p 2,j and j 2 be p 1,j 2/p 2,j 2 = in j S 2 p 1,j /p 2,j. By contradiction let us assue that p 1,j 2/p 2,j 2 < p 1,j 1/p 2,j 1. Let us denote by 1 the achine on which j 1 is scheduled, and 2 the achine on which j 2 is scheduled. Because the distributed DLB2C is running on all achines, it has been executed for all pairs of achines and does not change the solution (the distribution is stable), it has been executed to balance 1 and 2 in particular. As shown in the proof of Theore 6, there exists α such that j S( 1 ), j S( 2 ), p 1,j /p 2,j α and p 1,j /p 2,j α. DLB2C has a guaranteed perforance according to Theore 7 only if the schedule it generates is stable; that is to say when there are no ore pair wise exchange possible. The proof can be extended to the non stable cases as long as the achine i ax (as defined in the proof) can not exchange jobs with any other achine. Unfortunately, the algorith ight not converge. In the reainder of this section, we study the properties of the dynaic equilibriu of DLB2C using a arkovian odel and siulations. First we show the algorith ight not reach a static equilibriu. Proposition 8. In soe cases, DLB2C does not converge to a stable solution. Proof: Figure 1 presents an instance of the proble with 5 jobs and 3 achines organized in 2 clusters. The details of the instance is given in Figure 1(d). With a particular initial distribution of the jobs to the achines, the algorith is trapped in a cycle between the three schedules given in Figure 1(a), 1(b) and 1(c).

8 A B C (a) A B C (b) A B C (c) Job Cluster 1 = {A,B} Cluster 2 = {C} 1 4ɛ 1 2 4ɛ 1 3 ɛ 2ɛ 4 1 8ɛ ɛ (d) Cost of each job Figure 1. With the instance given in (d), DLB2C can get trappped in a cycle and never reach stability. Once in schedule (a) CLB2C will balance achines B and C and reach schedule (b). Then the only possible ove is to balance A and C which leads to schedule (c). Schedule (a) is then obtained by balancing A and B. A. Modelling the one cluster case We consider the use of algorith DLB2C on only one cluster of achines. Since there is only one cluster, the aount of work is fixed and equal to p i, and the longest task is of size p ax = ax p i. For the purpose of this section, we represent the state of the syste at any tie by an integer load vector L = {L 1,..., L }, L i being the load of achine i. A load vector is valid under two conditions j, L j and j L j = p i. The akespan of a load vector is siply the value of its largest diension C ax = ax L j. The DLB2C algorith can transfor a valid load vector L into another valid load vector L under the conditions: Only two achines are ipacted: j j 1, j 2, L j = L j The load is conserved: L j1 + L j2 = L j 1 + L j 2 The new ibalance is bounded: L j 1 L j 2 p ax This relation allows us to represent an instance of the one cluster case by a directed graph where the vertices are valid load vectors and where there is a directed edge between L and L if DLB2C can transfor L into L. Notice that this odel relaxes the notion of tasks in the sense that there is no ore task assignent but siply a load assignent. That is to say, there ight not exist an actual task assignent that aps to that particular load. However, one can easily convince herself that the following stays eaningful. i=1 A load vector is perfectly balanced if j, L j. Notice that all the perfectly balanced load vectors are equivalent up to a reordering of the diensions of the load vector. The directed graph constructed above can be decoposed in any strongly connected coponent. However, only one coponent has no outgoing edges, we call this coponent the sink coponent. Theore 9. The sink coponent is the only strongly connected coponent that does not have any outgoing edges. And this coponent contains the perfectly balanced state(s). Proof: First notice that all the perfectly balanced vectors are within the sae coponent since they have the sae nuber of diension with value and with value. One can go fro one perfectly balanced vector to an other one by appropriately balancing the offending diensions. Therefore, one can reduce all the perfectly balanced state to a single one. The theore is proven by showing that fro each load vector there is a path to the perfectly balanced state. Therefore, the sink coponent is the only one without outgoing edges. Let L be a load vector with C ax >. Let j ax be such that L jax = C ax and j in be such that L jin = in j L j. When j in and j ax are balanced, the load vector can becoe L with L Ljax +L j ax = jin 2, L j in = L jax + L jin L j ax and j j ax, j in L j = L j. If the akespan of L is, then the theore is proven. Otherwise, there are two cases: either the akespan strictly decreased ax j L j < C ax. Or it stayed the sae ax j L j = C ax; but in this case the nuber of values in L equal to C ax is one less than in L, by repeating this process, the akespan will eventually decrease. This creates a path fro L to a perfectly balanced load vector. Theore 1. All the load vectors in the sink coponent have a akespan saller or equal to p ax. Proof: Let us assue that X is the axiu akespan of a load vector L of the sink coponent. Because of the condition that the new ibalance is bounded, the relationship encoded by the edges of the graph there exist a achine j with a load greater than (or equal to) X p ax (could be ). If that achine could reach a higher load without taking work fro the achine that reaches the axiu akespan, X would not be axial. Therefore, we can recursively find achines with load equal to ax(x kp ax, ) for all k [.. 1]. But, 1 k= X kp ax thus X p ax. This analysis tells us that, eventually, the state of the syste will enter the sink coponent. Even if there exists states in that coponent with really large akespans, the syste can only reach such a state by carefully choosing pairs of achines to balance to reach this worst case scenario. In practice, the pairs of achines are chosen randoly and this worst case scenario is unlikely to be reached. We transfor the graph of states into a Markov chain

9 by introducing probabilities for transitioning fro one state to the other. We set the probabilities so that the rebalanced pair of achines j 1, j 2 is chosen uniforly. Once this pair is chosen and balanced, the reaining ibalance is also uniforly chosen in {,..., p ax }. Because there is only one sink coponent to the underlying graph, we can reduce the analysis to that coponent. The stationary distribution of the chain is well defined. We expanded the graph for various paraeters of, p ax and p i. We set p i so that the axiu ibalance given in Theore 1 can be reached. We nuerically coputed the stationary distribution of the Markov chain using an iterative ethod. With the stationary distribution, we coputed the probability distribution of the akespan in the steady state. The coputational cost quickly increases with and p ax, aking larger run prohibitively long. However we will see that we can safely extrapolate our conclusions fro relatively sall values. Figure 2 presents the distribution of akespan nuerically coputed for = 6 and various values of p ax and for p ax = 4 and various values of. The akespan is noralized so that the X-axis shows the deviation of the akespan fro the optially balanced case as a factor of p ax. It is striking that the distributions are uniodal and the ode is.5. All the distributions are qualitatively the sae. Increasing p ax only soothen the shape of the distribution (see Figure 2(a)). This allows us to confidently look at a reasonably sall value of p ax in Figure 2(b). When the nuber of achine increases, the probability that the akespan is before the ode decreases and that it is after the ode increases. However, in all coputed cases, C ax + 1.5p ax with very high probability. B. Siulation To perfor an analysis of the perforance of the algorith in the heterogeneous case and copare the to the one obtained in the hoogeneous one, we developed a siulator that ipleents CLB2C and DLB2C (Algoriths 5 and 7). We focus on two clusters of siilar size: there are often one or two CPU and one GPU per achine. As the clusters are coposed of hoogeneous achines, and because the clusters behave differently for each job, only the size of the cluster atter; The tie to execute a job on each cluster is a characteristic of the job itself. We randoly generate soe jobs with two arbitrary speeds (one for each cluster) and size, then we distribute the randoly aong the achines (details are given per experients). For each iteration of the siulation, a pair of achine is randoly chosen and then balanced using the decentralized algorith. As the partition can cycle, each siulation is stopped when the nuber of iterations is bigger than a threshold depending on the nuber of achines. We coputed the akespan of the solution obtained as well as the stability of the partition for two clusters of Probability Probability Noralized ibalance (cax-su/)/pax (a) = 6 pax=1 pax=2 pax=3 pax=4 pax=5 pax=6 pax=7 pax=8 pax= Noralized ibalance (cax-su/)/pax (b) pax = 4 =2 =3 =4 =5 =6 =7 =8 =9 Figure 2. Probability density function of the akespan. The x-axis presents noralized ibalance, i.e., C ax = + Xpax. 64 and 32 achines and for one hoogeneous cluster of 96 achines. For each experient, 768 jobs were used, they have length uniforely taken in the interval [1;1]. The results presented in Figure 3 show that the stability is rarely obtained in the heterogeneous case. However, if we look at the evolution of C ax over tie (Figure 4), we observe runs that quickly reach a value and oscillate around it. Even if there is no stability in the run, variations stay close to the iniu value the run has reached. We can see siilarities between the runs on an hoogeneous cluster and two different clusters: both have soe stable runs and soe cycles, they both quickly reach a akespan a lot saller than the initial one. The variations are ore intense on two clusters as there are ore possibilities for better solutions. But the hoogeneous and heterogeneous cases are qualitatively siilar. We now focus on the nuber of exchanges per achine needed to reach a akespan better than 15% of the akespan obtained with CLB2C (we denote this value as 1.5cent) for the first tie (Figure 5). This threshold is hopefully uch saller than 3 ties the optial akespan (CLB2C is a 2-approxiation). For all experients, the jobs were created using a unifor distribution on [1;1] to define their length. In the case of two clusters of 64 and 32 achines, 9% of runs reach this value in less than 4 exchanges per achines even if the nuber of jobs to balance increase greatly (for 1 to 32 jobs per achine). This also eans that this value is reach with less than 4

10 Fraction of stable siulations Nuber of jobs Figure 3. Stability of DLB2C on two clusters of size 64 and 32, length of each job taken uniforely on the interval [1;1]. Most instances do not reach a stable solution. Cax 6, 5,5 5, 4, Iterations 1 4 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 1 (a) 1 cluster of 96 achines. 1 different runs with identical paraeters: 768 jobs with a length uniforely taken between 1 and 1. Cax 4, 3,8 3,6 3,4 3,2 3, Iterations 1 4 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 1 (b) 2 clusters of 64 and 32 achines. 1 different runs with identical paraeters: 768 jobs with a length uniforely taken between 1 and 1. Figure 4. Evolution of C ax in ultiple siulations. The one cluster and two clusters cases do not show significant differences. counications per achine (about 4 counications). When the size of the cluster increases to 512 and 256 achines (8 ties ore), 9% of the achines reach 1.5cent for the first tie in less than 5 exchanges per achine. The shape of those results is siilar to the one obtained for one hoogeneous cluster of 96 achines, except for the fact that the initial distribution of jobs in the hoogeneous case is ore likely to be close to an optial situation: there are less possibilities for the algorith to optiize the distribution. VIII. CONCLUSION In this work we studied how to balance the load of an heterogeneous cluster using decentralized algoriths. Classical subission tie ethods and work stealing based ethods are unlikely to achieve desirable properties. Also general ethods are unlikely to be found, we investigated two different cases. MJTB is useful when the nuber of types of jobs is liited and has an approxiation ratio equal to the nuber of types of jobs considered. In the case of a cluster with two types of achines (such as a CPU-GPU cluster), we proposed DLB2C which is a 2- approxiation ratio if the algorith converges. However, we showed that it ight not converge and we studied its dynaic equilibriu using a Markov odel and siulations. The Markov odel provides a theoretical distribution of the akespan when there is only one cluster which highlights that the akespan tycally stays within a factor 2 of the optial. The siulation shows that the one cluster and two cluster case behave siilarly. It also shows that the two cluster cases reaches good solutions within a few iterations. Even without converging, DLB2C is a sensible algorith to use. Providing strong theoretical evidence that the distributions of the akespan obtained by DLB2C is suitable and its extension to ore than two clusters of achines are possible future works. Of course, designing another decentralized algorith that converges to a provably good solution reains our ain goal. Also the current odel ignores the aount of tasks exchanged; iniizing the nuber of tasks exchanged (or network usage) would certainly be of interest. REFERENCES [1] Michael A. Bender and Michael O. Rabin. Online scheduling of parallel progras on heterogeneous systes with applications to Cilk. Theory of Coputing Systes, Special Issue on SPAA, 35(3):289 34, 22. [2] P. Berenbrink, A. Brinkann, T. Friedetzky, and L. Nagel. Balls into non-unifor bins. In Proc. of IPDPS, 21. [3] Petra Berenbrink, André Brinkann, To Friedetzky, and Lars Nagel. Balls into non-unifor bins. J. Parallel Distrib. Coput., 74(2): , February 214. [4] Petra Berenbrink, Kayar Khodaoradi, Thoas Sauerwald, and Alexandre Stauffer. Balls-into-bins with nearly optial load distribution. In Proceedings of the Twenty-fifth Annual ACM Syposiu on Parallelis in Algoriths and Architectures, SPAA, pages , 213. [5] Robert D. Bluofe. Scheduling ultithreaded coputations by work stealing. In FOCS, pages , 1994.

11 Fraction of siulations where Cax 1.5cent jobs 192 jobs 384 jobs 768 jobs 1536 jobs 372 jobs Nuber of exchanges per achine Fraction of siulations where Cax 1.5cent jobs 192 jobs 384 jobs 768 jobs 1536 jobs 372 jobs Nuber of exchanges per achine Fraction of siulations where Cax 1.5cent jobs 1536 jobs 372 jobs 6144 jobs jobs jobs Nuber of exchanges per achine (a) 1 cluster of 96 achines (b) 2 clusters of 64 and 32 achines (c) 2 clusters of 512 and 256 achines Figure 5. Study of tie to reach a akespan less than 1.5cent. The nuber of iterations is noralized with respect to the nuber of achines. Most cases reaches the threshold within 5 pairwise exchanges per achine. [6] Robert D. Bluofe, Christopher F. Joerg, Bradley C. Kuszaul, Charles E. Leiserson, Keith H. Randall, and Yuli Zhou. Cilk: An efficient ultithreaded runtie syste. In Proceedings of the Fifth ACM SIGPLAN Syposiu on Principles and Practice of Parallel Prograing (PPoPP), page , [7] F. Warren Burton and M. Ronan Sleep. Executing functional progras on a virtual tree of processors. In Proceedings of the 1981 Conference on Functional Prograing Languages and Coputer Architecture, FPCA 81, pages , [8] Lin Chen, Deshi Ye, and Guochuan Zhang. Online scheduling on a CPU-GPU cluster. In T-H.Hubert Chan, LapChi Lau, and Luca Trevisan, editors, Theory and Applications of Models of Coputation, volue 7876 of Lecture Notes in Coputer Science, pages 1 9. Springer Berlin Heidelberg, 213. [9] Martin Gairing, Burkhard Monien, and Andrea Woclaw. Autoata, Languages and Prograing, volue 358, chapter A Faster Cobinatorial Approxiation Algorith for Scheduling Unrelated Parallel Machines, pages Springer, aug 25. [1] Michael R. Garey and David S. Johnson. Strong NPcopleteness results: Motivation, exaples, and iplications. J. ACM, 25(3):499 58, [11] Ronald L. Graha. Bounds for certain ultiprocessing anoalies. Bell Syste Technical Journal, 45: , [15] Dorit S. Hochbau and David B. Shoys. Using dual approxiation algoriths for scheduling probles: Practical and theoretical results. Journal of ACM, 34: , [16] Ralf Hoffann, Matthias Korch, and Thoas Rauber. Perforance evaluation of task pools based on hardware synchronization. In Proc. of Super Coputing, 24. [17] Ellis Horowitz and Sartaj Sahni. Exact and approxiate algoriths for scheduling nonidentical processors. J. ACM, 23(2): , April [18] Eugene L. Lawler and Jacques Labetoulle. On preeptive scheduling of unrelated parallel processors by linear prograing. J. ACM, 25(4): , [19] Victor W. Lee, Changkyu Ki, Jatin Chhugani, Michael Deisher, Daehyun Ki, Anthony D. Ngyuen, Nadathur Satish, Mikhail Selyanskiy, Srinivis Chennupaty, Per Haarlund, Ronak Singhal, and Pradeep Dubey. Debunking the 1x GPU vs. CPU yth: An evaluation of throughput coputing on CPU and GPU. In ISCA, 21. [2] Jan Karel Lenstra, David B. Shoys, and Éva Tardos. Approxiation algoriths for scheduling unrelated parallel achines. Matheatical Prograing, 46: , 199. [21] Marc Tchiboukdjian, Nicolas Gast, Denis Trystra, Jean- Louis Roch, and Julien Bernard. A tighter analysis of work stealing. In International Syposiu on Algoriths and Coputation (ISAAC), 21. [12] Ronald L. Graha. Bounds on ultiprocessing tiing anoalies. SIAM Journal of Applied Matheatics, 17(2): , [13] Ronald L. Graha, Eugene L. Lawler, Jan Karel Lenstra, and Alexander. H. G. Rinnooy Kan. Optiization and approxiation in deterinistic sequencing and scheduling: a survey. Annals of discrete atheatics, 5(2): , [14] Tiothy D. R. Hartley, Erik Saule, and Uit V. Catalyurek. Iproving perforance of adaptive coponentbased dataflow iddleware. Parallel Coputing, 38(6-7):289 39, 212.