SWITCH T F T F SELECT. (b) local schedule of two branches. (a) if-then-else construct A & B MUX. one iteration cycle

Transcription

1 768 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 Compile-ime Sceduling of Dynamic Constructs in Dataæow Program Graps Soonoi Ha, Member, IEEE and Edward A. Lee, Fellow, IEEE Abstract Sceduling dataæow graps onto processors consists of assigning actors to processors, ordering teir execution witin te processors, and specifying teir æring time. Wile all sceduling decisions can be made at runtime, te overead is excessive for most real systems. o reduce tis overead, compile-time decisions can be made for assigning andèor ordering actors on processors. Compile-time decisions are based on known proæles available for eac actor at compile time. e proæle of an actor suc as te execution time and te communication patterns. However, a dynamic construct witin a macro actor, suc as a conditional and a data-dependent iteration, makes te proæle of te actor unpredictable at compile time. For tose constructs, we propose to assume some proæle at compile-time and deæne a cost to be minimized wen deciding on te proæle under te assumption tat te runtime statistics are available at compile-time. Our decisions on te proæles of dynamic constructs are sown to be optimal under some bold assumptions, and expected to be near-optimal in most cases. e proposed sceduling tecnique as been implemented as one of te rapid prototyping facilities in Ptolemy. is paper presents te preliminary results on te performance wit syntetic examples. Keywords multiprocessor sceduling, dynamic constructs, proæle, macro actor, dataæow program graps. A I. Introduction D ataæow grap representation, eiter as a programming language or as an intermediate representation during compilation, is suitable for programming multiprocessors because parallelism can be extracted automatically from te representation ëë, ëë Eac node, or actor, in a dataæow grap represents eiter an individual program instruction or a group tereof to be executed according to te precedence constraints represented by arcs, wic also represent te æow of data. A dataæow grap is usually made ierarcical. In a ierarcical grap, an actor itself may represent anoter dataæow grap: it is called a macro actor. Particularly, we deæne a data-dependent macro actor, or data-dependent actor, as a macro actor of wic te execution sequence of te internal dataæow grap is data dependent ècannot be predicted at compile timeè. Some examples are macro actors tat contain dynamic constructs suc as conditional, data-dependent iteration, and recursion. Actors are said to be data-independent if not data-dependent. e sceduling task consists of assigning actors to processors, specifying te order in wic actors are executed on eac processor, and specifying te time at wic tey are S. Ha is wit te Department of Computer Engineering, Seoul National University, Seoul, 5-74, Korea. sa@comp.snu.ac.kr E. Lee is wit te Department of Electrical Engineering and Computer Science, University of California at Berkeley, Berkeley, CA 9470, USA. eal@om.eecs.berkeley.edu executed. ese tasks can be performed eiter at compile time or at run time ë3ë. In te fully-dynamic sceduling, all sceduling decisions are made at run time. It as te æexibility to balance te computational load of processors in response to canging conditions in te program. In case a program as a large amount of non-deterministic beavior, any static assignment of actors may result in very poor load balancing or poor sceduling performance. en, te fully dynamic sceduling would be desirable. However, te run-time overead may be excessive; for example it may be necessary to monitor te computational loads of processors and sip te program code between processors via networks at run time. Furtermore, it is not usually practical to make globally optimal sceduling decision at run time. In tis paper, we focus on te applications wit a moderate amount of non-deterministic beavior suc as DSP applications and grapics applications. en, te more sceduling decisions are made at compile time te better in order to reduce te implementation costs and to make it possible to reliably meet any timing constraints. Wile compile-time processor sceduling as a very ric and distinguised istory ë4ë, ë5ë, most eæorts ave been focused on deterministic models: te execution time of eac actor i on a processor P k is æxed and tere are no data-dependent actors in te program grap. Even in tis restricted domain of applications, algoritms tat accomplis an optimal sceduling ave combinatorial complexity, except in certain trivial cases. erefore, good euristic metods ave been developed over te years ë4ë, ë6ë, ë7ë, ë8ë. Also, most of te sceduling tecniques are applied to a completely expanded dataæow grap and assume tat an actor is assigned to a processor as an indivisible unit. It is simpler, owever, to treat a data-dependent actor as a scedulable indivisible unit. Regarding a macro actor as a scedulable unit greatly simpliæes te sceduling task. Prasanna et al ë9ë scedule te macro dataæow graps ierarcically to treat macro actors of matrix operations as scedulable units. en, a macro actor may be assigned to more tan one processor. erefore, new sceduling tecniques to treat a macro actor as a scedulable unit was devised. Compile-time sceduling assumes tat static information about eac actor is known. We deæne te proæle of an actor as te static information about te actor necessary for a given sceduling tecnique. For example, if we use a list sceduling tecnique, te proæle of an actor is simply te computation time of te actor on a processor. e communication requirements of an actor wit oter actors are included in te proæle if te sceduling tec-

2 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS 769 nique requires tat information. e proæle of a macro actor would be te number of te assigned processors and te local scedule of te actor on te assigned processors. For a data-independent macro actor suc as a matrix operation, te proæle is deterministic. However, te proæle of a data-dependent actor of dynamic construct cannot be determined at compile time since te execution sequence of te internal dataæow subgrap varies at run time. For tose constructs, we ave to assume te proæles someow at compile-time. e main purpose of tis paper is to sow owwe can deæne te proæles of dynamic constructs at compile-time. A crucial assumption we rely on is tat we can approximate te runtime statistics of te dynamic beavior at compiletime. Simulation may be a proper metod to gater tese statistics if te program is to be run on an embedded DSP system. Sometimes, te runtime statistics could be given by te programmer for grapics applications or scientiæc applications. By optimally coosing te proæle of te dynamic constructs, we will minimize te expected scedule lengt of a program assuming te quasi-static sceduling. In ægure, actor A is a data-dependent actor. e sceduling result is sown wit a gantt cart, in wic te orizontal axis indicates te sceduling time and te vertical axis indicates te processors. At compile time, te proæle of actor A is assumed. At run time, te scedule lengt of te program varies depending on te actual beavior of actor A. Note tat te pattern of processor availability before actor B starts execution is preserved at run time by inserting idle time. en, after actor A is executed, te remaining static scedule can be followed. is sceduling strategy is called quasi-static sceduling tat was ærst proposed by Lee ë0ë for DSP applications. e strict application of te quasistatic sceduling requires tat te syncronization between actors is guaranteed at compile time so tat no run-time syncronization is necessary as long as te pattern of processor availability is consistent wit te sceduled one. It is generally impractical to assume tat te exact run-time beaviors of actors are known at compile time. erefore, syncronization between actors is usually performed at run time. In tis case, it is not necessary to enforce te pattern of processor availability by inserting idle time. Instead, idle time will be inserted wen syncronization is required to execute actors. Wen te execution order of te actors is not canged from te sceduled order, te actual scedule lengt obtained from run-time syncronization is proven to be not muc diæerent from wat te quasi-static sceduling would produce ë3ë. Hence, our optimality criterion for te proæle of dynamic constructs is based on te quasi-static sceduling strategy, wic makes analysis simpler. II. Previous Work All of te deterministic sceduling euristics assume tat static information about te actors is known. But almost none ave addressed ow to deæne te static information of data-dependent actors. e pioneering work on tis issue was done by Martin and Estrin ëë. ey calculated S S A C (a) B D S A C (b) A B S A B C D C D (c) (d) Fig.. èaè A dataæow grap consists of æve actors among wic actor A is a data-dependent actor. èbè Gantt cart for compile-time sceduling assuming a certain execution time for actor A. ècè At run time, if actor A takes longer, te second processor is padded wit no-ops and èdè if actor A takes less, te ærst processor is idled to make te pattern of processor availability same as te sceduled one èdark lineè in te quasi-static sceduling. te mean pat lengt from eac actor to a dummy terminal actor as te level of te actor for list sceduling. For example, if tere are two possible pats divided by a conditional construct from an actor to te dummy terminal actor, te level of te actor is a sum of te pat lengts weigted by te probability wit wic te pat is taken. us, te levels of actors are based on statistical distribution of dynamic beavior of data-dependent actors. Since tis is expensive to compute, te mean execution times instead are usually used as te static information of data-dependent actors ëë. Even toug te mean execution time seems a reasonable coice, it is by no means optimal. In addition, bot approaces ave te common drawback tat a datadependent actor is assigned to a single processor, wic is a severe limitation for a multiprocessor system. wo groups of researcers ave proposed quasi-static sceduling tecniques independently: Lee ë0ë and Loeæer et al ë3ë. ey developed metods to scedule conditional and data-dependent iteration constructs respectively. Bot approaces allow more tan one processor to be assigned to dynamic constructs. Figure sows a conditional and compares tree sceduling metods. In ægure èbè, te local scedules of bot brances are sown, were two brances are sceduled on tree processors èn = 3è wile te total number of processors is 4 è = 4è. In Lee's metod, we overlap te local scedules of bot brances and coose te maximum termination for eac processor. For ard real-time systems, it is te proper coice. Oterwise, it may be ineæcient if eiter one branc is more likely to be taken and te size of te likely branc is muc smaller. On te oter and, Loeæer takes te local scedule of more likely branc as te proæle of te conditional. is strategy is ineæcient if bot brances are equally likely to be taken and te size of te assumed branc ismuc larger. Finally, a conditional evaluation can be replaced wit a conditional assignment to make te construct static; te grap is modiæed as illustrated in ægure ècè. In tis sceme, bot true and false brances are sced- B D

3 770 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 SWICH F A A & B B F SELEC (a) if-ten-else construct 3 A MUX (c) a fully-static scedule N B A 3 (b) local scedule of two brances MAX(A,B) (d) Lee s metod 3 3 B p(a) > p(b) (e) Leoffler s metod Fig.. ree diæerent scedules of a conditional construct. èaè An example of a conditional construct tat forms a data-dependent actor as a wole. èbè Local deterministic scedules of te two brances. ècè A static scedule by modifying te grap to use conditional assignment. èdè Lee's metod to overlap te local scedules of bot brances and to coose te maximum for eac processor. èeè Loeæer's metod to take te local scedule of te branc wic is more likely to be executed. uled and te result from one branc is selected depending on te control boolean. An immediate drawback is ineæciency wic becomes severe wen one of te two brances is not a small actor. Anoter problem occurs wen te unselected branc generates an exception condition suc as divide-by-zero error. All tese metods on conditionals are ad-oc and not appropriate as a general solution. Quasi-static sceduling is very eæective for a datadependent iteration construct if te construct can make eæective use of all processors in eac cycle of te iteration. It scedules one iteration and pads wit no-ops to make te pattern of processor availability at te termination te same as te pattern of te start èægure 3è èequivalently, all processors are occupied for te same amount of time in eac iterationè. en, te pattern of processor availability after te iteration construct is independent of te number of iteration cycles. is sceme breaks down if te construct cannot utilize all processors eæectively. one iteration cycle Fig. 3. A quasi-static sceduling of a data-dependent iteration construct. e pattern of processor availability is independent of te number of iteration cycles. e recursion construct as not yet been treated successfully in any statically sceduled data æow paradigm. Recently, a proper representation of te recursion construct as been proposed ë4ë. But, it is not explained ow toscedule te recursion construct onto multiprocessors. Wit ænite resources, careless exploitation of te parallelism of te recursion construct may cause te system to deadlock. In summary, dynamic constructs suc as conditionals, data-dependent iterations, and recursions, ave not been treated properly in past sceduling eæorts, eiter for static sceduling or dynamic sceduling. Some ad-oc metods ave been introduced but proven unsuitable as general solutions. Our earlier result wit data-dependent iteration ë3ë demonstrated tat a systematic approac to determine te proæle of data-dependent iteration actor could minimize te expected scedule lengt. In tis paper, we extend our analysis to general dynamic constructs. In te next section, we will sowow dynamic constructs are assigned teir proæles at compile-time. We also prove te given proæles are optimal under some unrealistic assumptions. Our experiments enable us to expect tat our decisions are near-optimal in most cases. Section 4,5 and 6 contains an example wit data-dependent iteration, recursion, and conditionals respectively to sow ow te proæles of dynamic constructs can be determined wit known runtime statistics. We implement our tecnique in te Ptolemy framework ë5ë. e preliminary simulation results will be discussed in section 7. Finally, we discuss te limits of our metod and mention te future work. III. Compile-ime Profile of Dynamic Constructs Eac actor sould be assigned its compile-time proæle for static sceduling. Assuming a quasi-static sceduling strategy, te proposed sceme is to decide te proæle of a construct so tat te average scedule lengt is minimized assuming tat all actors except te dynamic construct are data-independent. is objective is not suitable for a ard real-time system as it does not bound te worst case beavior. We also assume tat all dynamic constructs are uncorrelated. Wit tis assumption, we may isolate te effect of eac dynamic construct on te scedule lengt separately. In case tere are inter-dependent actors, we may group tose actors as anoter macro actor, and decide te optimal proæle of te large actor. Even toug te decision of te proæle of te new macro actor would be complicated in tis case, te approac is still valid. For nested dynamic constructs, we apply te proposed sceme from te inner dynamic construct ærst. For simplicity, all examples in tis paper will ave only one dynamic construct in te dataæow grap. e run-time cost of an actor i, C i, is te sum of te total computation time devoted to te actor and te idle time due to te quasi-static sceduling strategy over all processors. In ægure, te run-time cost of a data-dependent actor A is te sum of te ligtly ècomputation timeè and darkly saded areas after actor A or C èimmediate idle time after te dynamic constructè. e scedule lengt of a certain iteration can be written as scedule lengt = èc i + Rè; were is te total number of processors in te system, and R is te rest of te computation including all idle time tat may result bot witin te scedule and at te end. erefore, we can minimize te expected scedule lengt by minimizing te expected cost of te data-dependent ac- èè

4 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS 77 tor or dynamic construct if we assume tat R is independent of our decisions for te proæle of actor i. is assumption is unreasonable wen precedence constraints make R dependent on our coice of proæle. Consider, for example, a situation were te dynamic construct is always on te critical pat and tere are more processors tan we can eæectively use. en, our decision on te proæle of te construct will directly aæect te idle time at te end of te scedule, wic is included in R. On te oter and, if tere is enoug parallelism to make eæective use of te unassigned processors and te execution times of all actors are small relative to te scedule lengt, te assumption is valid. Realistic situations are likely to fall between tese two extremes. o select te optimal compile-time proæle of actor i, we assume tat te statistics of te runtime beavior is known at compile-time. e validity of tis assumption varies to large extent depending on te application. In digital signal processing applications were a given program is repeatedly executed wit data stream, simulation can be useful to obtain te necessary information. In general, owever, we may use a well-known distribution, for example uniform or geometric distribution, wic makes te analysis simple. Using te statistical information, we coose te proæle to give te least expected cost at runtime as te compile-time proæle. e proæle of a data-dependent actor is a local scedule wic determines te number of assigned processors and computation times taken on te assigned processors. e overall algoritm of proæle decision is as follows. We assume tat te dynamic beavior of actor i is expressed wit parameter k and its distribution pèkè. min = LARGE; èè is te total number of processors. èè N is te number of processors assigned to te actor. forn=tof èè AèN,kè is te actor cost wit parameter N, k èè pèkè is P te probability of parameter k temp = k pèkèaèn; kè; if ètemp é minè min = temp; g In te next section, we will illustrate te proposed sceme wit data-dependent iteration, recursion, and conditionals respectively to sow ow proæles are decided wit runtime statistics. IV. Data Dependent Iteration In a data-dependent iteration, te number of iteration cycles is determined at runtime and cannot be known at compile-time. wo possible dataæow representations for data-dependent iteration are sown in ægure 4 ë0ë. e numbers adjacent to te arcs indicate te number of tokens produced or consumed wen an actor æres ëë. In ægure 4 èaè, since te upsample actor produces M tokens eac time it æres, and te iteration body consumes only one token wen it æres, te iteration body must ære M times for eac æring of te upsample actor. In ægure 4 èbè, te source of M UPSAMPLE M Iteration body M DOWNSAMPLE (a) F SELEC f SWICH F (b) Fig. 4. Data-dependent iteration can be represented using te eiter of te dataæow graps sown. e grap in èaè is used wen te number of iterations is known prior to te commencement of te iteration, and èbè is used oterwise. number of iterations need not be known prior to te commencement of te iteration. Here, a token coming in from above is routed troug a "select" actor into te iteration body. e "D" on te arc connected to te control input of te "select" actor indicates an initial token on tat arc wit value "false". is ensures tat te data coming into te "F" input will be consumed te ærst time te "select" actor æres. After tis ærst input token is consumed, te control input to te "select" actor will ave value "true" until function tèè indicates tat te iteration is ænised by producing a token wit value "false". During te iteration, te output of te iteration function fèè will be routed around by te "switc" actor, again until te test function tèè produces a token wit value "false". ere are many variations on tese two basic models for data-dependent iteration. e previous work ë3ë considered a subset of datadependent iterations, in wic simultaneous execution of successive cycles is proibited as in ægure 4 èbè. In ægure 4 èaè, tere is no suc restriction, unless te iteration body itself contains a recurrence. erefore, we generalize te previous metod to permit overlapped cycles wen successive iteration cycles are invokable before te completion of an iteration cycle. Detection of te intercycle dependency from a sequential language is te main task of te parallel compiler to maximize te parallelism. A dataæow representation, owever, reveals te dependency rater easily wit te presence of delay on a feedback arc. We assume tat te probability distribution of te number of iteration cycles is known or can be approximated at compile time. Let te number of iteration cycles be a random variable I wit known probability mass function pèiè. For simplicity, we set te minimum possible value of I to be 0. We let te number of assigned processors be N and te total number of processors be. We assume a blocked scedule as te local scedule of te iteration body to remove te unnecessary complexity inall illustrations, altoug te proposed tecnique can be applicable to te overlap execution scedule ë6ë. In a blocked scedule, all assigned processors are assumed to be available, or syncronized at te beginning. us, te execution time of one iteration cycle wit N assigned processors is t N as displayed in ægure 5 èaè. We denote by s N te time tat must t D

5 77 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 elapse in one iteration before te next iteration is enabled. is time could be zero, if tere is no data dependency between iterations. Given te local scedule of one iteration cycle, we decide on te assumed number of iteration cycles, x N, and te numberofoverlapped cycles k N. Once te two parameters, x N and k N, are cosen, te proæle of te data-dependent iteration actor is determined as sown in ægure 5 èbè. e subscript N of t N, s N, x N and k N represents tat tey are functions of N, te number of te assigned processors. For brevity, we will omit te subscript N for te variables witout confusion. Using tis proæle of te data-dependent macro actor, global sceduling is performed to make a ierarcical compilation. Note tat te pattern of processor availability after execution of te construct is diæerent from tat before execution. We do not address ow to scedule te iteration body in tis paper since it is te standard problem of static sceduling. N kn s t next iteration cycle is executable (a) (c) i < x i x kn (b) i = x x (d) i > x x Fig. 5. èaè A blocked scedule of one iteration cycle of a datadependent iteration actor. A quasi-static scedule is constructed using a æxed assumed number x of cycles in te iteration. e cost of te actor is te sum of te dotted area èexecution timeè and te dark area èidle time due to te iterationè. ere displays 3 possible cases depending on te actual number of cycles i in èbè for i = x, ècè for iéx, and èdè for iéx. According to te quasi-static sceduling policy, tree cases can appen at runtime. If te actual number of cycles coincides wit te assumed number of iteration cycles, te iteration actor causes no idle time and te cost of te actor consists only of te execution time of te actor. Oterwise, some of te assigned processors will be idled if te iteration takes fewer tan x cycles èægure 5 ècèè, or else te oter processors as well will be idled èægure 5 èdèè. e expected cost of te iteration actor, CèN; k; xè, is a sum of te individual costs weigted by te probability mass function of te number of iteration cycles. e expected cost becomes xx CèN; k; xè = pèièntx + pèièèntx + i,x td k eè: i=x+ èè By combining te ærst term wit te ærst element of te second term, tis reduces to CèN; k; xè =Ntx + t i pèièd, x e: è3è k i=x+ i Our metod is to coose tree parameters èn, k, and xè in order to minimize te expected cost in equation è3è. First, we assume tat N is æxed. Since CèN; k; xè is a decreasing function of k wit æxed N, we select te maximum possible number for k. e number k is bounded by two ratios: N and t s. e latter constraint is necessary to avoid any idle time between iteration cycles on a processor. As a result, k is set to be k = minèb N c; b t s cè: e next step is to determine te optimal x. Ifavalue x is optimal, te expected cost is not decreased if we vary x by + or,. erefore, we obtain te following inequalities, CèN; k; xè, CèN; k; x +è=,nt+t pèx ++ikè ç 0 CèN; k; xè, CèN; k; x, è = Nt,t pèx + ikè ç 0: Since t is positive, from inequality è5è, pèx ++ikè N ç ç pèx + ikè: If k is equal to, te above inequality becomes same as inequality è5è in ë3ë, wic sows tat te previous work is a special case of tis more general metod. Up to now, we decided te optimal value for x and k for a given number N. How tocoose optimal N is te next question we ave to answer. Since t is not a simple function of N, no closed form for N minimizing CèN; k; xè exists, unfortunately. However, we may use exaustive searc troug all possible values N and select te value minimizing te cost in polynomial time. Moreover, our experiments sow tat te searc space for N is often reduced signiæcantly using some criteria. Our experiments sow tat te metod is relatively insensitive to te approximated probability mass function for i ë3ë. Using some well-known distributions wic ave nice matematical properties for te approximation, we can reduce te summation terms in è3è and è6è to closed forms. Let us consider a geometric probability mass function wit parameter q as te approximated distribution of te number of iteration cycles. is models a class of asymmetric bell-saped distributions. e geometric probability mass function means tat for any non-negativeinteger r, P ëj ç rë =q r ; Pëj=rë=pèrè=q r è, qè: o use inequality è6è, we ænd è4è è5è è6è è7è q x pèx + ikè =è,qè,qk: è8è

6 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS 773 erefore, from te inequality è6è, te optimal value of x satisæes, q, q k qx+ N ç ç, q, q k qx : è9è Using æoor notation, we can obtain te closed form for te optimal value as follows: x = blog q Nè, q k è è, qè c: Furtermore, equation è3è is simpliæed by using te fact getting i=x+ è0è i pèièd, x k e = qx+, q ; èè k CèN; k; xè =Ntx + t qx+, q k : èè Now, we ave all simpliæed formulas for te optimal proæle of te iteration actor. Similar simpliæcation is possible also wit uniform distributions ë7ë. If k equals to, our results coincide wit te previous result reported in ë3ë. V. Recursion Recursion is a construct wic instantiates itself as a part of te computation if some termination condition is not satisæed. Most ig level programming languages support tis construct since it makes a program compact and easy to understand. However, te number of self-instantiations, called te dept of recursion, is usually not known at compile-time since te termination condition is calculated at run-time. In te dataæow paradigm, recursion can be represented as a macro actor tat contains a SELF actor èægure 6è. A SELF actor simply represents an instance of a subgrap witin wic it sits. If te recursion actor as only one SELF actor, te function of te actor can be identically represented by a datadependent iteration actor as sown in ægure 4 èbè in te previous section. is includes as a special case all tail recursive constructs. Accordingly, te sceduling decision for te recursion actor will be same as tat of te translated data-dependent iteration actor. In a generalized recursion construct, we mayave more tan one SELF actor. e number of SELF actors in a recursion construct is called te widt of te recursion. In most real applications, te widt of te recursion is no more tan two. A recursion construct wit widt and dept is illustrated in ægure 6 èbè and ècè. We assume tat all nodes of te same dept in te computation tree ave te same termination condition. We will discuss te limitation of tis assumption later. We also assume tat te run-time probability mass function of te dept of te recursion is known or can be approximated at compile-time. e potential parallelism of te computation tree of a generalized recursion construct may be uge, since all nodes at te same dept can be executed concurrently. e maximum degree of parallelism, owever, is usually not known at compile-time. Wen we exploit te parallelism of te construct, we sould consider te resource limitations. We may ave to restrict te parallelism in order not to deadlock te system. Restricting te parallelism in case te maximum degree of parallelism is too large as been recognized as a diæcult problem to be solved in a dynamic dataæow system. Our approac proposes an eæcient solution by taking te degree of parallelism as an additional component of te proæle of te recursion construct. Suppose tat te widt of te recursion construct is k. Let te dept of te recursion be a random variable I wit known probability mass function pèiè. We denote te degree of parallelism by d, wic means tat te descendents at dept d in te computation grap are assigned to different processor groups. A descendent recursion construct at dept d is called a ground construct èægure 7 èaèè. If we denote te size of eac processor group by N, te total number of processors devoted to te recursion becomes Nk d. en, te proæle of a recursion construct is deæned by tree parameters: te assumed dept of recursion x, te degree of parallelism d, and te size of a processor group N. Our approac optimizes te parameters to minimize te expected cost of te recursion construct. An example of te proæle of a recursion construct is displayed in ægure 7 èbè. Let ç be te sum of te execution times of actors, and in ægure 6. And, let ç o be te sum of te execution times of actors a and b. en, te scedule lengt l x of a ground construct becomes l x = çèk 0 +k +æææ+k x,d, è+ç o k x,d = ç kx,d, k, +ç ok x,d ; è3è wen x ç d. At run time, some processors will be idled if te actual dept of recursion is diæerent from te assumed dept of recursion, wic is illustrated in ægure 7 ècè and èdè. Wen te actual dept of recursion i is smaller tan te assumed dept x, te assigned processors are idled. Oterwise, te oter processors as well are idled. Let R be te sum of te execution times of te recursion besides te ground constructs. is basic cost R is equal to Nçèkd,è k,. For i ç x, te runtime cost, C, becomes C = R + Nk d èç kx,d, k, +ç o k x,d è; è4è assuming tat x is not less tan d. For iéx, te cost C becomes C = R + Nk d èç ki,d, +ç o k i,d è+ k, è,nk d ç èè k, + ç oèèk i,d, k x,d è: è5è erefore, te expected cost of te recursion construct, CèN; x; dè is te sum of te run-time cost weigted by te probability mass function. xx CèN; d; xè =R+ pèièc + pèièc : i=x+ è6è

7 774 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 SELF actor f a SWICH F test dept 0 c a a a a function f(x) a(x); if test(x) is RUE return b(x); else return (f(c(x)),f(c(x))); f f F SELEC b b b b b 0 (a) (b) (c) Fig. 6. èaè An example of a recursion construct and èbè its dataæow representation. e SELF actor represents te recursive call. ècè e computation tree of a recursion construct wit two SELF actors wen te dept of te recursion is two. l x N 3 4 ground construct Nk d 3 4 (a) (b) (c) (d) Fig. 7. èaè e reduced computation grap of a recursion construct of widt wen te degree of parallelism is. èbè e proæle of te recursion construct. e scedule lengt of te ground construct is a function of te assumed dept of recursion x and te degree of parallelism d. A quasi-static scedule is constructed depending on te actual dept i of te recursion in ècè for iéxand in èdè for iéx. After a few manipulations, CèN; d; xè = Nèç kx, k, + ç ok x è+ ç pèiè è k, + ç oèèk i,d, k x,d è7è è: i=x+ First, we assume tat N is æxed. Since te expected cost is a decreasing function of d, we select te maximum possible number for d. e number d is bounded by te processor constraint: Nk d ç. Since we assume tat te assumed dept of recursion x is greater tan te degree of parallelism d, te optimal value for d is d = minèblog k N c;xè: è8è Next, we decide te optimal value for x from te observation tat if x is optimal, te expected cost is not decreased wen x is varied by + and,. erefore, we get CèN; d; xè, CèN; d; x +è= èç+ç o èk,èèè,nk x +k x,d pèièè ç 0 i=x+ CèN; d; xè, CèN; d; x, è =

8 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS 775 èç + ç o èk, èèènk x,,k x,d, pèièè ç 0: è9è i=x Rearranging te inequalities, we get te following, i=x+ pèiè ç Nkd ç i=x pèiè: è0è Note te similarity of inequality è0è wit tat for datadependent iterations è6è. In particular, if k is, te two formulas are equivalent as expected. e optimal values d and x depend on eac oter as sown in è8è and è0è. We may need to use iterative computations to obtain te optimal values of d and x starting from d = blog c. k N Let us consider an example in wic te probability mass function for te dept of te recursion is geometric wit parameter q. At eac execution of dept i of te recursion, we proceed to dept i + wit probability q and return to dept i, wit probability,q.from te inequality è0è, te optimal x satisæes As a result, x becomes q x+ ç Nkd ç qx : èè Nk d x = blog q c: èè Up to now, we assume tat N is æxed. Since ç is a transcendental function of N, te dependency of te expected cost upon te size of a processor group N is not clear. Instead, we examine te all possible values for N, calculate te expected cost from equation è3è and coose te optimal N giving te minimum cost. e complexity of tis procedure is still polynomial and usually reduced signiæcantly since te searc space of N can be reduced by some criteria. In case of geometric distribution for te dept of te recursion, te expected cost is simpliæed to CèN; d; xè =Nèç kx, k, +ç ok x x,d qx+ è+ èç+ç o èk,èèk, qk : è3è In case te number of cild functions is one èk = è, our simpliæed formulas wit a geometric distribution coincide wit tose for data-dependent iterations, except for an overead term to detect te loop termination. Recall tat our analysis is based on te assumption tat all nodes of te same dept in te computation tree ave te same termination condition. is assumption rougly approximates a more realistic assumption, wic we call te independence assumption, tat all nodes of te same dept ave equal probability of terminating te recursion, and tat tey are independent eac oter. is equal probability is considered as te probability tat all nodes of te same dept terminate te recursion in our assumption. e expected number of nodes at a certain dept is te same in bot assumptions even toug tey describe different beaviors. Under te independence assumption, te sape of te proæle would be te same as sown in ægure 7: te degree of parallelism d is maximized. Moreover, all recursion processors ave te same scedule lengt for te ground constructs. However, te optimal scedule lengt l x of te ground construct would be diæerent. e lengt l x is proportional to te number of executions of te recursion constructs inside a ground construct. is number can be any integer under te independence assumptions, wile it belongs to a subset f0;k ;k ; :::g under our assumption. Since te probability mass function for tis number is likely to be too complicated under te independence assumption, we sacriæce performance by coosing a sub-optimal scedule lengt under a simpler assumption. VI. Conditionals Decision making capabilityisan indispensable requirement of a programming language for general applications, and even for signal processing applications. A dataæow representation for an if-ten-else and te local scedules of bot brances are sown in ægure èaè and èbè. We assume tat te probability p wit wic te "RUE" branc èbranc è is selected is known. e "FALSE" branc èbranc è is selected wit probability p =,p. Let t ij be te ænising time of te local scedule of te i-t branc on te j-t processor. And let ^tj be te ænising time on te j-t processor in te optimal proæle of te conditional construct. We determine te optimal values f^tj g to minimize te expected runtime cost of te construct. Wen te i-t branc is selected, te cost becomes NX k= NX ^t k + maxë0; max jë;në èt ij, ^tj èë: X erefore, te expected cost CèN è is CèNè= ^t k + k= i= p i maxë0; max jë;në èt ij, ^tj èë: è4è è5è It is not feasible to obtain te closed form solutions for ^tj because te max function is non-linear and discontinuous. Instead, a numerical algoritm is developed.. Initially, take te maximum ænis time of bot branc scedules for eac processor according to Lee's metod ë0ë.. Deæne æ i = maxë0; max j èt ij, ^tj è; 0èë. Initially, all æ i =0. e variable æ i represents te excessive cost per processor over te expected cost wen branc i is selected at run time. We deæne te bottleneck processors of branc i as te processors fjg tat satisfy te relation t ij, ^tj = æ i. For all brances fig, repeat te next step. 3. Coose te set of bottleneck processors, æ i, of branc i only. If we decrease ^tj by æ for all j æ i, te variation of te expected cost becomes æcèn è = è,jæ i j + p i èæ. Increase æ until te set æ i needs to be updated. Update æ i and repeat step 3. Now, we consider te example sown in ægure. Suppose p =0:3 and p =0:7. e initial proæle in our algoritm is same as Lee's proæle as sown in ægure 8 èaè, wic

9 776 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 appens to be same as Loeæer's proæle in tis speciæc example. e optimal proæle determined by our algoritm is displayed in ægure 8 èbè. N 3 3 (a) initial profile (b) optimal profile Fig. 8. Generation of te optimal proæle for te conditional construct in ægure. èaè initial proæle èbè optimal proæle. We generalized te proposed algoritm to te M-way branc construct by case construct. o realize an M-way branc, we prefer to using case construct to using a nested if-ten-else constructs. Generalization of te proposed algoritm and proof of optimality is beyond te scope of tis paper. For te detailed discussion, refer to ë7ë. For a given number of assigned processors, te proposed algoritm determines te optimal proæle. o obtain te optimal number of assigned processors, we compute te total expected cost for eac number and coose te minimum. VII. An Example e proposed tecnique to scedule data-dependent actors as been implemented in Ptolemy, wic is a eterogeneous simulation and prototyping environment being developed in U.C.Berkeley, U.S.A. ë5ë. One of te key objectives of Ptolemy is to allow many diæerent computational models to coexist in te same system. A domain is a set of blocks tat obey a common computational model. An example of mixed domain system is sown in ægure 9. e syncronous dataæow èsdfè domain contains all dataindependent actors and performs compile-time sceduling. wo brances of te conditional constructs are represented as SDF subsystems, so teir local scedules are generated by a static sceduler. Using te local scedules of bot brances, te dynamic dataæowèddfè domain executes te proposed algoritm to obtain te optimal proæle of te conditional construct. e topmost SDF domain system regards te DDF domain as a macro actor wit te assumed proæle wen it performs te global static sceduling. SDF DDF SDF SWICH F F SELEC Fig. 9. An example of mixed domain system. e topmost level of te system is a SDF domain. A dynamic constructèif-ten-elseè is in te DDF domain, wic in turn contains two subsystems in te SDF domain for its brances. SDF We apply te proposed sceduling tecnique to several syntetic examples as preliminary experiments. ese experiments do not serve as a full test or proof of generality of te tecnique. However, tey verify tat te proposed tecnique can make better sceduling decisions tan oter simple but ad-oc decisions on dynamic constructs in many applications. e target arcitecture is assumed to be a sared bus arcitecture wit 4 processors, in wic communication can be overlapped wit computation. o test te eæectiveness of te proposed tecnique, we compare it wit te following sceduling alternatives for te dynamic constructs. Metod. Assign all processors to eac dynamic construct Metod. Assign only one processor to eac dynamic construct Metod 3. Apply a fully dynamic sceduling ignoring all overead Metod 4. Apply a fully static sceduling Metod corresponds to te previous researc on quasistatic sceduling tecnique made by Lee ë0ë and by Loefæer et. al. ë3ë for data dependent iterations. Metod approximately models te situation wen we implement eac dynamic construct as a single big atomic actor. o simulate te tird model, we list all possible outcomes, eac of wic can be represented as a data-independent macro actor. Wit eac possible outcome, we replace te dynamic construct wit a data-independent macro actor and perform fully-static sceduling. e sceduling result from Metod 3 is non-realistic since it ignores all te overead of te fully dynamic sceduling strategy. Noneteless, it will give a yardstick to measure te relative performance of oter sceduling decisions. By modifying te dataæow graps, we may use fully static sceduling in Metod 4. For a conditional construct, we mayevaluate bot brances and select one by amultiplexor actor. For a data-dependent iteration construct, we always perform te worst case number of iterations. For comparison, we use te average scedule lengt of te program as te performance measure. As an example, consider a For construct of datadependent iteration as sown in ægure 0. e number inside eac actor represents te execution lengt. o increase te parallelism, we pipelined te grap at te beginning of te For construct. e sceduling decisions to be made for te For construct are ow many processors to be assigned to te iteration body and ow many iteration cycles to be sceduled explicitly. We assume tat te number of iteration cycles is uniformly distributed between and 7. o determine te optimal number of assigned processors, we compare te expected total cost as sown in table I. Since te iteration body can utilize two processors eæectively, te expected total cost of te ærst two columns are very close. However, te scedule determines tat assigning one processor is sligtly better. Rater tan parallelizing te iteration body, te sceduler automatically parallelizes te iteration cycles. If we cange te parameters, we maywant to parallelize te iteration body ærst and te iteration cycles next.

10 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS FOR UP- SAMPLE body DOWN- SAMPLE Fig. 0. An example wit a For construct at te top level. e subsystems associated wit te For construct are also displayed. e number inside an actor represents te execution lengt of te actor. e proposed tecnique considers te tradeoæs of parallelizing inner loops or parallelizing outer loops in a nested loop construct, wic as been te main problem of parallelizing compilers for sequential programs. e resulting Gantt cart for tis example is sown in ægure FOR Fig.. A Gantt cart disply of te sceduling result over 4 processors from te proposed sceduling tecnique for te example in ægure 0. e proæle of te For construct is identiæed. If te number of iteration cycles at run time is less tan or equal to 3, te scedule lengt of te example is same as te scedule period 66. If it is greater tan 3, te scedule lengt will increase. erefore, te average scedule lengt of te example becomes e average scedule lengt from oter sceduling decisions are compared in table II. e proposed tecnique outperforms oter realistic metods and acieves 85è of te ideal scedule lengt by Metod 3. In tis example, assigning 4 processors to te iteration body èmetod è worsens te performance since it fails to exploit te intercycle parallelism. Conæning te dynamic construct in a single actor èmetod è gives te worst performance as expected since it fails to exploit bot intercycle parallelism and te parallelism of te iteration body. Since te range of te number of iteration cycles is not big, assuming te worst case iteration èmetod 4è is not bad. is example, owever, reveals a sortcoming of te proposed tecnique. If we assign processors to te iteration body to exploit te parallelism of te iteration body as well as te intercycle parallelism, te average scedule lengt becomes 77.7, wic is sligtly better tan te sceduling result by te proposed tecnique. Wen we calculate te 4 expected total cost to decide te optimal number of processors to assign to te iteration body, we do not account for te global eæect of te decision. Since te diæerence of te expected total costs between te proposed tecnique and te best sceduling was not signiæcant, as sown in table I, tis non-optimality of te proposed tecnique could be anticipated. o improve te performance of te proposed tecnique, we can add a euristic tat if te expected total cost is not signiæcantly greater tan te optimal one, we perform sceduling wit tat assigned number and compare te performance wit te proposed tecnique to coose te best sceduling result. e searc for te assumed number of iteration cycles for te optimal proæle is not faultless eiter, since te proposed tecnique ænds a local optimum. e proposed tecnique selects 3 as te assumed number of iteration cycles. It is proved, owever, tat te best assumed number is in tis example even toug te performance diæerence is negligible. Altoug te proposed tecnique is not always optimal, it is certainly better tan any ofte oter sceduling metods demonstrated in table II. Experiments wit oter dynamic constructs as well as nested constructs ave been performed to obtain te similar results tat te proposed tecnique outperforms oter ad-oc decisions. e resulting quasi-static scedule could be at least 0è faster tan oter sceduling decisions currently existent, wile it is as little as 5 è slower tan an ideal èand igly unrealisticè fully-dynamic scedule. In a nested dynamic construct, te compile-time proæle of te inner dynamic construct aæects tat of te outer dynamic construct. In general, tere is a trade-oæ between exploiting parallelism of te inner dynamic construct ærst and tat of te outer construct ærst. e proposed tecnique resolves tis conæict automatically. Refer to ë7ë for detailed discussion. Let us assess te complexity of te proposed sceme. If te number of dynamic constructs including all nested ones is D and te number of processors is N, te total number of proæle decision steps is order of ND, OèNDè. o determine te optimal proæle also consumes OèNDè time units. erefore, te overall complexity is order of ND. e memory requirements are te same order od magnitude as te number of proæles to be maintained, wic is also order of ND. VIII. Conclusion As long as te data-dependent beavior is not dominating in a dataæow program, te more sceduling decisions are made at compile time te better, since we can reduce te ardware and software overead for sceduling at run time. For compile-time decision of task assignment andèor ordering, we need te static information, called proæles, of all actors. Most euristics for compile-time decisions assume te static information of all tasks, or use ad-oc approximations. In tis paper, we propose a systematic metod to decide on proæles for eac dynamic construct. We deæne te compile-time proæle of a dynamic construct as an assumed local scedule of te body of te dynamic

11 778 IEEE RANSACIONS ON COMPUERS, VOL. 46, NO. 7, JULY 997 ABLE I e expected total cost of te For construct as a function of te number of assigned processors Number of assigned processors 3 4 Expected total cost NèA ABLE II Performance comparison among several sceduling decisions Metod Proposed 3 4 Average scedule lengt è of ideal construct. We deæne te cost of a dynamic construct and coose its compile-time proæle in order to minimize te expected cost. e cost of a dynamic construct is te sum of execution lengt of te construct and te idle time on all processors at run-time due to te diæerence between te compile-time proæle and te actual run-time proæle. We discussed in detail ow to compute te proæle of tree kinds of common dynamic constructs: conditionals, datadependent iterations, and recursion. o compute te expected cost, we require tat te statistical distribution of te dynamic beavior, for example te distribution of te number of iteration cycles for a datadependent iteration, must be known or approximated at compile-time. For te particular examples we used for experiments, te performance does not degrade rapidly as te stocastic model deviates from te actual program beavior, suggesting tat a compiler can use fairly simple tecniques to estimate te model. We implemented te tecnique in Ptolemy as a part of a rapid prototyping environment. We illustrated te eæectiveness of te proposed tecnique wit a syntetic example in tis paper and wit many oter examples in ë7ë. e results are only a preliminary indication of te potential in practical applications, but tey are very promising. Wile te proposed tecnique makes locally optimal decisions for eac dynamic construct, it is sown tat te proposed tecnique is eæective wen te amount of data dependency from a dynamic construct is small. But, we admittedly cannot quantify at wat level te tecnique breaks down. Acknowledgments e autors would like to gratefully tank te anonymous reviewers for teir elpful suggestions. is researc is part of te Ptolemy project, wic is supported by te Advanced Researc Projects Agency and te U.S. Air Force èunder te RASSP program, contract F C-37è, te State of California MICRO program, and te following companies: Bell Nortern Researc, Cadence, Dolby, Hitaci, Lucky-Goldstar, Mentor Grapics, Mitsubisi, Motorola, NEC, Pilips, and, Rockwell. References ëë W.B.Ackerman, "Data Flow Languages", Computer, Vol. 5, ëë ë3ë ë4ë ë5ë ë6ë ë7ë ë8ë ë9ë No., pp. 5-5, Feb. 98. E.A.Lee and D.G.Messerscmitt, "Syncronous Data Flow", IEEE Proceedings, September, 987. S.Ha and E.A.Lee, "Compile-ime Sceduling and Assignment of Dataæow Program Graps wit Data-Dependent Iteration", IEEE rans. Computers, November, 99. E.G.Coæman,Jr., Computer and Job Sceduling eory, Wiley, New York, 976. M.J.Gonzalez, "Deterministic Processor Sceduling", Computing Surveys, 9è3è, September, 977. G.C.Si, "Multiprocessor Sceduling to Account for Interprocessor Communications", P.D. esis, University of California, Berkeley, April, 99. S.J.Kim and J.C.Browne, "A General Approac to Mapping of Parallel Computations Upon Multiprocessor Arcitecture", Int. Conf. on Distributed Computing Systems, pp. -8, 988. K.Konstantinides, R..Kanesiro, and J.R.ani, "ask Allocation and Sceduling Models for Multiprocessor Digital Signal Processing", IEEE rans. Acoustics, Speec, and Signal Processing, Vol.38, No., pp. 5-6, December, 990. Computing Surveys, 9è3è, September, 977. G.N.S.Prasanna, A.Agarwal, and B.R.Musicus, "Hierarcical Compilation of Macro Dataæow Graps for Multiprocessors wit Local Memory", IEEE rans. on Parallel and Distributed Systems, Vol.5, No.7, pp , July, 994. ë0ë E.A.Lee, "Recurrences, Iteration, and Conditionals in Statically Sceduled Block Diagram Languages", VLSI Signal Processing III, IEEE Press, 988. ëë D.F.Martin and G.Estrin, "Pat Lengt Computation on Grap Models of Computations", IEEE rans. on Computers, C-8, pp , June, 969. ëë M. Granski, I. Korn, and G.M.Silberman, "e Eæect of Operation Sceduling on te Performance of a Data Flow Computer", IEEE rans. on Computers, C-36è9è, September, 987. ë3ë C.Loeæer, A.Ligtenberg, H.Beda, and G. Moscytz, "Hierarcical Sceduling Systems for Parallel Arcitectures", Proceedings of Euco, Grenoble, September, 988. ë4ë P.A.Suler, J.Biswas, K.M.Korner, and J.C.Browne, "DFL: A ask-level Dataæow Language", Journal of Parallel and Distributed Computing, 9, pp. 03-5, 990. ë5ë J. Buck, S. Ha, E. A. Lee, and D.G. Messerscmitt, "Ptolemy: A Framework for Simulating and Prototyping Heterogeneous Systems", International Journal of Computer Simulation Vol. 4. April, pp. 55-8, 994. ë6ë P.Hoang and J.Rabaey, "Program Partitioning for a Reconægurable Multiprocessor System", IEEE Worksop on VLSI Signal Processing IV, November, 990. ë7ë S. Ha, "Compile-ime Sceduling of Dataæow Program Graps wit Dynamic Constructs," P.D. dissertation, University of California, Berkeley, May, 99.

12 HA AND LEE: COMPILE-IME SCHEDULING OF DYNAMIC CONSRUCS IN DAAFLOW PROGRAM GRAPHS 779 Soonoi Ha Soonoi Ha is an assistant professor in te Computer Engineering Department at Seoul National University, Seoul, Korea. His researc interests include arcitecture and sceduling tecniques for parallel processing, design metodology for digital systems, and ardware-software codesign. He as been actively involved in te Ptolemy project since its birt. He received te B.S. and te M.S. degrees in electronics from Seoul National University, Seoul, Korea, in 985 and 987, respectively. He received te P.D. degree in te Electrical Engineering and Computer Science Department at U.C. Berkeley in 99. He is a member of ACM and te IEEE Computer Society. Edward A. Lee Edward A. Lee is a Professor in te Electrical Engineering and Computer Science Department at U.C. Berkeley. His researc interests include real-time software, discrete-event systems, parallel computation, arcitecture and software tecniques for signal processing, and design metodology for eterogeneous systems. He is director of te Ptolemy project at UC Berkeley. He is co-autor of four books, numerous papers, and two patents. His bacelors degree èb.s.è is from Yale University è979è, is masters ès.m.è from MI è98è, and is P.D. from U. C. Berkeley è986è. From 979 to 98 e was a member of tecnical staæ at Bell elepone Laboratories in Holmdel, New Jersey, in te Advanced Data Communications Laboratory. He is a founder of Berkeley Design ecnology, Inc. and as consulted for a number of oter companies. He is a fellow of te IEEE and was an NSF Presidential Young Investigator.