A Class of Generalized Stochastic Petri Nets for the Performance Evaluation of Multiprocessor Systems

Transcription

1 A Class of Generalized Stochastic Petri Nets for the Performance Evaluation of Multiprocessor Systems MARCO AJMONE MARSAN and GIANNI CONTE Politecnico di Torino, Turin, Italy and GIANFRANCO BALBO Universita' di Torino, Turin, Italy Generalized Stochastic Petri Nets (GSPNs) are presented and are applied to the performance evaluation of multiprocessor systems. GSPNs are derived from standard Petri nets by partitioning the set of transitions into two subsets comprising timed and immediate transitions. An exponentially distributed random firing time is associated with each timed transition, whereas immediate transitions fire in zero time. It is shown that GSPNs are equivalent to continuous-time stochastic processes, and solution methods for the derivation of the steady state probability distribution are presented. Examples of application of GSPN models to the performance evaluation of multiprocessor systems show the usefulness and the effectiveness of this modeling tool. 1. INTRODUCTION Graph models have been proposed by many authors as a useful tool for the analysis of peculiar features of computer systems such as concurrency, synchronization, communication, and cooperation among subsystems. Much of the work in this field is related to original ideas developed by C. A. Petri in his Ph.D. dissertation [12]. These graph models are today generally known as Petri Nets (PNs). A PN comprises a set of places P, a set of transitions T, and a set of directed arcs A. In the graphical representation of PNs places are drawn as circles and transitions as bars. Arcs connect transitions to places and places to transitions. Places may contain tokens, which are drawn as black dots. The state of a PN is defined by the number of tokens contained in each place and is denoted by a This work was financially supported by the CNR Computer Science Program, MUMICRO contract , and by NATO Research Grant Authors' addresses: M. Ajmone Marsan and G. Conte, Dipartimento di Elettronica, Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129, Torino, Italy; G. Balbo, Dipartimento di Informatica, Universita' di Torino, Corso Massimo d' Azeglio 42, 10125, Torino, Italy. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission ACM /84/ $00.75 ACM Transactions on Computer Systems, Vol. 2, No. 2, May 1984, Pages

2 94 M. Ajmone Marsan, G. Balbo and G. Conte vector M, whose ith component represents the number of tokens in the ith place. The PN state is usually called the PN marking. The definition of a PN requires the specification of the initial marking M'. A formal definition of a PN is thus the following [11]: PN = (P, T, A, M) P= {pl, p2,...,p,} T = {tl, t2..., tin} Ac {px T}U {TxP} M' = {m;, m~..., m~'} A place is an input to a transition if an arc exists from the place to the transition. A place is an output from a transition if an arc exists from the transition to the place. A transition is enabled when all of its input places contain at least one token. Enabled transitions can fire, thus removing one token from each input place and placing one token in each output place. Each firing of a transition modifies the distribution of tokens on places and thus produces a new marking for the PN. The firing of a transition may disable other transitions that are said to be in conflict with the one that fires. In a PN with a given initial marking M ', the reachability set is defined as the set of all markings that can be "reached" from M' by means of a sequence of transition firings. If the PN is such that, for any possible marking, the maximum number of tokens in any place is less than or equal to k, then the PN is said to be k-bounded. The widespread acceptance that PNs have gained in the modeling of computer systems is due to the fact that they are very well-suited for the representation of concurrency and of synchronization. With PNs the logical behavior of a system is described by explicitly defining the causal relationships existing between events and conditions. PN models are used to investigate the presence of desirable or undesirable properties in a system (e.g., deadlocks). No time-related performance measure can be obtained by means of a PN model since time is not considered in a PN. PNs have been the subject of intensive research efforts aimed at obtaining a complete understanding of the basic properties of the model and at investigating their applicability to the representation of real systems. Dealing with practical applications, several authors have found it necessary to extend and/or modify the basic model definitions to obtain more convenient modeling tools. In particular, the possibility of representing in the model the time involved in system operations is an extension to which much attention has been devoted. One of the first efforts in this direction is represented by E-nets [10]. The main difference between E-nets and PNs is the former's introduction of a fixed time associated with each transition to specify the delay between the enabling and the firing. Other authors have recently extended the PN models by including time as a specification. Zuberek [15] used a fixed firing time in order to model the behavior of a computer system at a very low level. Merlin and Farber [6] introduced timed PNs for the analysis of protocols, associating with each transition a maximum (1)

3 A Class of Generalized Stochastic Petri Nets 95 and a minimum firing time. Stochastic PN models in which a random firing time is associated with each transition were independently proposed by Symons, Natkin, and Molloy [13, 8, 9]. The stochastic PNs (SPNs) described by M. K. Molloy [7] are obtained by associating with each transition in a PN an exponentially distributed firing time. A formal definition of a SPN is thus the following. where P, T, A, and M' are as in (1), and SPN = (P, T, A, M', R), (2) R = {rl, r2,..., rm} (3) is the set of firing rates (possibly marking-dependent) associated with the PN transitions. Molloy [7] has shown that SPNs are isomorphic to continuous time Markov chains (MCs) due to the memoryless property of the exponential distribution of firing times. SPN markings correspond to MC states. In particular, k-bounded PNs are isomorphic to finite MCs. The sojourn time in each marking (state) is an exponentially distributed random variable with average r] where H is the set of transitions that are enabled by the marking. The transition rate from marking Mi to marking Mj is obtained as 2 rk, (5) k~hij where Hij is the set of transitions (which contains only one element in most of the eases) enabled by marking Mi whose firing generates marking Mj. The infinitesimal generator Q of the MC isomorphic to the SPN can thus be easily obtained, and the steady state probability distribution of the MC, P, is computed by solving the usual matrix equation: PQ = O. (6) Molloy has also shown that with appropriate series and parallel expansions, using the method of stages [5], it is possible to model transitions with generally distributed random firing times, provided these distributions have rational Laplace transforms. With the introduction of SPNs a link is established between two important classes of computer system models: the above-mentioned graph models, and the probabilistic models familiar to performance analysts. SPNs are a very useful tool for the analysis of computer systems since they allow the system operations to be precisely described by means of a graph that translates into a Markovian model useful for obtaining performance estimates. A very important feature of SPNs is that they can be understood by people who are not familiar with the probabilistic modeling approach, and that the derivation of the MC model and its solution can be made automatic, and thus transparent to the user.

4 96 M. Ajmone Marsan, G. Balbo and G. Conte The limitation of SPNs is that the graphical representation of systems rapidly becomes more difficult as system size and complexity increase. Moreover the number of states of the associated MC grows very fast as the dimensions of the graph increase. SPNs can thus be used to model only systems of limited size. Often it is not desirable to associate a random time with each transition, since one would rather associate times only with the events believed to have the largest impact on system performance. A typical example is the case in which the operating sequence of a system comprises activities whose durations differ by orders of magnitude. It is then conceivable to model the brief activities only from the logical point of view, whereas time is associated with the longer lasting ones. This choice becomes particularly convenient if by doing so the number of states of the associated MC model is reduced, thereby reducing the solution complexity. Generalized SPNs (GSPNs) presented in this paper contain two types of transitions: timed and immediate. An exponentially distributed firing time is associated only with timed transitions, since immediate transitions, by definition, fire in zero time. It is shown that GSPNs are still equivalent to Markovian models, and that their solution can be obtained with much less effort than is necessary for SPNs. Examples of GSPN model application to the performance analysis of multiprocessor systems are presented. 2. GENERALIZED STOCHASTIC PETRI NETS GSPNs are obtained by allowing transitions to belong to two different classes: immediate transitions and timed transitions. Immediate transitions fire in zero time once they are enabled. Timed transitions fire after a random, exponential y distributed enabling time. In the figures in this paper we use the Convention of drawing timed transitions as thick bars, and immediate transitions as thin bars. Firing rates are obviously associated only with timed transitions, and they may depend on the GSPN marking. A formal definition of a GSPN is thus as in (2), where the array R now contains only m' elements, m' being the number of timed transitions in the GSPN. Several transitions may be simultaneously enabled by a marking. If the Set of enabled transition H comprises only timed transitions, then the enabled timed transition ti (i E H) fires with probability ri (7) Y~ rk ' hell exactly as with SPNs. If H comprises both immediate and timed transitions, then only immediate transitions can fire. If H comprises zero or more timed transitions and only one immediate transition, then this is the one that fires. When H comprises several immediate transitions it is necessary to specify a proability density function on the set of enabled immediate transitions according to which the firing transition is selected. The subset of H comprising all enabled immediate transitions together with the associated probability distribution is called a random switch. This associated probability distribution will be called a Switching distribution. Different markings may originate a single random switch whenever they enable the same set of immediate transitions upon which a single A(~M Transactions on Computer Systems, Vol. 2, No. 2, May 1984.

5 A Class of Generalized Stochastic Petri Nets 97 (possibly marking-dependent) switching distribution can be defined. If the probability associated with an enabled immediate transition is zero, the transition cannot fire, and thus behaves as if it were not enabled. This condition, in some cases, can be modeled in a simpler way by means of inhibitor arcs. An inhibitor arc connects a place to a transition and is represented by a line terminating with a circle rather than with an arrowhead at the transition. The firing condition of a transition can thus be generalized by saying that a transition is enabled when all of its "normal" input places contain at least one token and no token is contained in the transition "inhibitor inputs." The firing of the transition removes a token from each of its normal input places. Inhibitor arcs do not expand the modeling power of GSPNs, but in some cases, as we shall see in a later section, they allow a simpler description of the system operations, since the use of inhibitor arcs may reduce the number of random switches to be defined in the GSPN. 3. AN EXAMPLE OF GSPN Consider, for example, the GSPN of Figure 1, which comprises seven places and seven transitions. Three of the transitions are timed: tl, t4, and ts. Transition t~ fires at a marking-dependent rate equal to u times the number of tokens in place pl. Transitions t4 and t5 fire at fixed rates v and z, respectively. Transitions t6 and t7 are two conflicting immediate transitions: they are always enabled simultaneously, so that it is necessary to define a switching distribution for each marking in which m7 is larger than zero. The immediate transitions t2 and t3 may be simultaneously enabled if P3 and p4 contain tokens. A switching distribution must hence be defined for each marking in which m2, m3, and m4 are greater than zero. Note that no more than two immediate transitions can be simultaneously enabled. Two random switches can thus be identified in this GSPN. A possible switching distribution definition is the one given in Table I. Starting from the initial GSPN marking shown in Figure 1, a possible evolution of the GSPN state may be the following: after an exponentially distributed random time with average 1/2u, transition t~ fires, and one of the two tokens contained in p~ moves to P2. Now in zero time either t2 or t3 fires. The firing transition is selected according to the switching distribution defined in Table i, which in this case assigns equal probabilities to the two transitions. Assume that 6 fires, removing a token from P2 and P3, and placing one in Ps. The two timed transitions h and t4 are now enabled. Transition t~ fires with probability whereas transition t4 fires with probability u (8) U+V' v (9) u+v" If tl fires first, one token moves from p, to P2, thus enabling the immediate transition t3. This transition fires immediately, being the only enabled transition of this type, thus moving one token from p2 to P6 and removing one token from P4. The resulting GSPN marking is such that one token exists in places p~ and

6 98 M. Ajmone Marsan, G. Balbo and G. Conte Pt t~ m~u Fig. 1. Example of GSPN. Table I. Switching Probabilities of the GSPN in Figure 1 t Pr(t2) m3 m3 + m4 Pr(t3) - m m3 + m4 Pr(tD m3 + m4 if m3 # 0 and m, # 0 Pr(t7) = m~ mz + m4 Pr(te) = Pr(tT) = 1/2 if m3 = m4 = 0 p~. The enabled transitions are t4, and ts, each of which can fire first with the following probabilities. v z Pr(t4) -- v + z' Pr(t5) - v + z (10)

7 A Class of Generalized Stochastic Petri Nets 99 Table II. Reachability Set of the GSPN in Figure 1 ml m2 m3 m4 ms ms m7 Markings that enable only timed transitions Markings that enable immediate transitions Assume t4 fires, so that a token is moved from p5 to pt, and a token is put in Pl. The two immediate transitions to t6 and t7 are now simultaneously enabled, and, as specified by the switching distribution defined in Table I, each of them can fire with probability 1/2, so that the token in p7 can move either to p3 or to p4. Now transitions tl and t~ are enabled, and the PN evolution continues. It must be pointed out that in general the reachability set of a GSPN is a subset of the reachability set of the associated PN, because precedence rules introduced with immediate transitions do not allow some states to be reached. The reachability set of a SPN is, instead, the same as for the associated PN. Table II shows the reachability set of the GSPN in Figure 1. It comprises 16 markings, whereas the teachability set of the associated PN comprises 33 states. Moreover, the reachability set of the GSPN can be divided into two disjoint subsets, one of which comprises markings that enable timed transitions only, while the other comprises markings that enable immediate transitions. The definition of random switches in a GSPN may sometimes require ingenuity and insight in the system operations. Consider, for instance, the GSPN in Figure 1 with the marking {1, 1, 1, 1, 0, 0, 0}. The semantic of the enabled random switch (will t2 or t3 fire first?) is implicit in some well-defined local property of the system. So it is easy to identify the probabilities associated with the two events. Consider now the portion of GSPN drawn in Figure 2. Assume tl (t2) fires first, so that a token moves to place p~ (P2), thus enabling the immediate transitions t4 and t~ (t6 and tt). The switching distribution on the two enabled transitions can be easily defined because it depends on some local and already identified behavior of the system. Assume now that t3 fires first, so that a token is placed in both pl and p2, thus simultaneously enabling the four immediate

8 100 M. Ajmone Marsan, G. Balbo and G. Conte "-[- "-I- "-1- "-t- Fig. 2. Portion of a GSPN model with multiple random switches. STATE y TIME Fig. 3- Possible sample path of the GSPN underlying point process presenting "multiple discontinuities." transitions t4, ts, t6, and t7. In this case the random switch accounts for the (possibly unclear) relations existing between the local behaviors of two separate parts of the system. A crucial aspect of the definition of a GSPN is the identification of all random switches and the definition for all possible cases of the "correct" switching distributions. 4. EVALUATION OF THE GSPN STEADY STATE PROBABILITY DISTRIBUTION By examining GSPN behavior as a function of time we can easily realize that it is equivalent to the time behavior of a stochastic point process (SPP) IX(t), t _> 01 with a finite state space. A one-to-one correspondence exists between GSPN markings and SPP states. In the remainder of this paper, the terms marking and state are used equivalently. Sample functions, such as the one depicted in Figure 3, representing a possible behavior sequence of the SPP, may present "multiple discontinuities" due to the sequential firing of one or more immediate transitions. The process is observed to spend a nonnegative amount of time in markings enabling timed transitions only, while it transits in zerotime through markings enabling immediate transitions. We call a state (or

9 A Class of Generalized Stochastic Petri Nets 101 marking) of the former type tangible and a state (or marking) of the latter type vanishing. The GSPN application purposes that are considered in this paper allow the following assumptions to be made. (1) The teachability set is finite. (2) Firing rates do not depend on time. (3) The initial marking is reachable with a nonzero probability from any marking in the teachability set. No marking (or group of markings) exists that "absorbs" the process. These assumptions further specify the nature of the SPP that can thus be classified as a finite state space, and a stationary (homogeneous), irreducible, and continuous-time stochastic point process. Disregarding, for the time being, the concept of time, and focusing our attention on the set of states that the process enters because of a transition out of a given state, we can observe that a stationary embedded Markov chain (EMC) can be recognized within the SPP. The behavioral specifications listed above are sufficient for the computation of the transition probabilities (note that the word "transition" in this context indicates a change of state). Let S indicate the state space of the EMC (and of the SPP as well), and let us distinguish tangible and vanishing states within S so that it can be partitioned in the following way: S = state space of SPP; IS[ =/(8 T = set of tangible states in SPP; V = set of vanishing states in SPP; TAV=0, S=TuV, K~=K~+K~. [ T [ = Kt [ V[ -- Kv The transition probability matrix U of the EMC can be written as follows. ~-Ko-'-* [ ~'Kt"'~ ] ~-.-Kv----~ [ ~-'Kr--~ C D 0 0 Kv U=A +B= + "~ 0 0 E F Kt (11) The elements of matrix A can be obtained using the characteristics of random switches, and the elements of matrix B can be obtained using the firing rates of timed transitions as in eq. (7). The solution of the system of linear equations, Y = YU, (12)

10 102 M. Ajmone Marsan, G. Balbo and G. Conte in which Y is a row vector representing the steady state probability distribution of the EMC, can be interpreted in terms of numbers of transitions performed by the EMC, observing that 1 - = E[number of transitions performed by the EMC to return to state i]. yi Loosely speaking, 1/yi can thus be recognized to be the mean recurrence time for state i (measured in number of transitions). Selecting one of the states of the EMC as a reference state for the chain, and assuming, without loss of generality, that it is a tangible state (say state i), it is easy to show that V u = Y--; (13) Yi represents the mean number of visits to state j between two subsequent visits to state i. The computation of the steady state probability distribution of the SPP can be obtained reintroducing the concept of time by means of the average sojourn time in each state (STi, i E S). Using the definition of immediate transitions, or equivalently that of vanishing states, we can write: ST~ = 0, i E V. (14) The computation of the average sojourn times in tangible states requires the definition of the set so that Hi = {(timed) transitions enabled by the tangible state i}, i E T, 1 - STi, 7, r: i E T. (15) /ehi The mean amount of time spent by the SPP to return to reference state i (mean cycle time) is thus given by Wi = VuSTi = Z YuSTj, (16) j~s jet where VuST; is the mean time spent by the SPP in state j during a cycle. The average fraction of time spent by the SPP in each of its states can be computed using renewal cycle arguments such as VijSTj Pi- Wi ' j E S. (17) The steady state probability distribution of the SPP can be finally written as o jev, P; = VuST ~ (18) -~i jet.

11 A Class of Generalized Stochastic Petri Nets 103 Given the conditions specified at the beginning of this section, it can be shown that this solution method is equivalent to a simple extension of the method originally proposed by Molloy [7, 8] to handle similar cases. Let us assume that all immediate transitions are replaced by timed transitions characterized by very high firing rates proportional to an arbitrary value x. Under this assumption all states are tangible, so that the GSPN reduces to a standard SPN and Molloy's solution method applies. If an explicit solution expression for the probability distribution of this standard SPN (expressed in terms of x) is obtained, the stationary probability distribution of the original GSPN can be obtained by taking the limit for x going to infinity of this solution. Since most practical cases involve GSPNs with a large state space, an explicit expression of the solution in terms of x is usually not easy to obtain and the practical approach suggested by Molloy of numerically solving the problem by assuming x to be very large and arbitrarily setting those probabilities that appear exceptionally small to zero is prone to numerical problems. 5. A COMPUTATIONALLY MORE EFFICIENT SOLUTION METHOD The solution method proposed in the previous section is computationally acceptable whenever the size of the set of vanishing states is small (compared with the size of the set of tangible states, i.e., Kv << K~). However this method requires the computation of the mean number of visits to each vanishing state that, as we know a priori, does not increase the information content of the solution. Moreover, the vanishing states not only require useless computations, but, by enlarging the size of the transition probability matrix U, make computation of the visits expensive and in some cases even impossible to obtain. In order to remove the vanishing states from the EMC introduced in the previous section, and to define a reduced embedded Markov chain (REMC) over tangible states only, we must compute the total transition probabilities among tangible states only. For the purpose of the derivation of a method for accomplishing this task, let i and j represent arbitrary tangible states (i, j E T), while r and s represent arbitrary vanishing states (r, s E V). Using c... dry, el~, and [ij to represent elements of the submatrix blocks C, D, E, and F of the transition probability matrix U of the original EMC, the total transition probability matrix between any two tangible states i and j (u[j) can be computed in the following way: U[j = /ij "at- Y~ eir Pr[r --)j ], (19) re V where Pr[r ~ j] represents the probability that the SPP moves from vanishing state r to tangible state j in an arbitrary number of steps, following a path through vanishing states only. In the Appendix we show that the transition probability matrix U' of the REMC can be obtained in a computationally efficient way using the expression U' = F + EG ~, (20) where each component of G = is the explicit expression of the above-mentioned Pr[r --) j ].

12 104 M. Ajmone Marsan, G. Balbo and G. Conte LMI I LM 2 I" i LM 3 ir I I J L... J L -~--- J "> LB, GB1 Fig. 4. Multiprocessor architecture. The solution of the problem Y' = Y'U' (21) in which Y' is a row vector representing the steady state probability distribution of the REMC, allows the direct computation of the mean number of visits performed by the SPP to tangible states only between two subsequent visits to a reference state. The stationary probability distribution associated with the set of tangible states is thus readily obtained by means of their average sojourn times as discussed in the previous section. The advantage of this method of solution is twofold. First, the time and space complexity of the solution is reduced, since instead of solving a system of K~ linear equations we must now (in the worst case) compute the inverse of a Kv Kv matrix and then solve a system of Kt linear equations. Recalling that the complexity of the Gauss elimination solution method of systems of K linear equations is 0 (K3), the proposed approach reduces the complexity of the solution of the GSPN from O(K~) to O(K3t) + O(K~). Second, by decreasing the impact of the size of the set of vanishing states on the complexity of the solution method, we are allowed a greater freedom in the explicit specification of the logical conditions of the original GSPN, making it easier to understand. 6. APPLICATION OF GSPNS TO THE PERFORMANCE ANALYSIS OF MULTIPROCESSOR SYSTEMS As an application of GSPNs to the performance evaluation of multiprocessor computer systems, consider the architecture shown in Figure 4. A set of p processing units, each comprising a CPU and a private memory, cooperate by exchanging messages through a set of m common memories that can be reached through an interconnection network of b buses. The performance of such a multiprocessor system is constrained by many different causes, among which one may mention the overhead introduced by the supervisory program, the presence of synchronization primitives among concurrent processes, and the contention for the use of global system resources. The latter phenomenon involves the physical structure of the multiprocessor system,

13 A Class of Generalized Stochastic Petri Nets 105 hence the architecture design. Causes of performance degradation at this level are very difficult to remove at higher levels. Models of contention for physical resources in multiprocessor systems are thus very important in order to obtain quantitative performance estimates that can be used in the architecture design stage together with cost considerations. In order to obtain a Markovian model of the multiprocessor operations, we assume that processors execute in their private memory for an exponentially distributed random time with average 1/h before issuing an access request directed to one of the common memories in the system. The request may not be immediately served, either because no bus is available, or because the addressed memory is busy. The durations of accesses to common memories are independent, exponentially distributed, random variables with average 1/~. The following bus arbitration policy is assumed: if a processor, say A, is in queue for a memory already accessed by another processor, say B, then at the end of the access the bus is not released but is immediately given to processor A. The key phenomenon that we want to model is the competition for physical resources. In order to simplify the model we assume that the times required for bus arbitration and release are negligible with respect to access times, queue times, and execution times. We can thus build a GSPN model using timed transitions to represent the issuing and the service of requests, and immediate transitions to represent the bus arbitration and its release. Figure 5 depicts a model of a system with 5 processors, 3 common memories, and 2 buses in which processors and memories are assumed to behave in a statistically equivalent manner. Note that this model can accommodate memories with different characteristics, whereas processors with different behavioral patterns induce a substantial modification of the GSPN. The GSPN comprises 21 places, 16 timed transitions, and 15 immediate transitions. Tokens in place Pl represent processors that are executing in their private memory. Tokens in P2 represent buses available for use. Tokens in P4, P,o, and P16 represent idle common memory modules. Each processor may issue an access request at a rate X so that tl fires at rate m~ ~,. A token in P3 represents a processor that needs to select a memory to perform an external access. If no bus is available the processor waits; if a bus is available the processor selects a memory according to a uniform distribution. Because of the symmetry of the GSPN we explain its behavior by focusing our attention on the encased subnet describing the access to the first common memory. Assume that one bus is available and a processor in P3 decides to access the first common memory. Two cases arise. The first case corresponds to an access to an idle memory module. Transition t2 is the only enabled transition that fires, removing tokens from p2, P3 and P4 and placing a token in Ps, meaning that a bus is used by a processor accessing a common memory that becomes busy. Upon the firing of transition t2 the memory access immediately begins, and timed transition t7 starts counting. The second case corresponds to an access request directed to a busy common memory. Up to four processors can be queued in front of each common memory at the time an access request is issued. This is represented in the GSPN of Figure 5 by a token in places ps, P6, P7 and ps. Notice that only one of these places can be occupied by a token at any given time. Let us suppose that a token is in place

14 106 M. Ajmone Marsan, G. Balbo and G. Conte COMMON MEMORY 2 PI P1 P1 PlO ~ t17 ~ tle ~ t19 P1 P1 t20 ~ t21 P~ - - P2 t. tto~ P1 t91 f~ t28 f~ o ~ ted ~J t71 ~J L._...) P, ml~ m P~ ~/~o ~ P~ I \ \ Fig. 5. GSPN model of the multiprocessor architecture shown in Figure 4 assuming that the system comprises 5 processors, 3 memories, and 2 buses. Due to the symmetry of the system, accesses to the 3 common memories are represented by 3 subnets. p~ to represent that one processor is accessing the common memory and no other processor is waiting in the queue for this same common memory. Transition t3 is enabled and fires, moving a token one step up in the chain of places Ps-P9 (from place P5 to place p6). The bus used to begin this (unsuccessful) access is immediately released. Because of the firing of transition t3, the counting of transition t7 is stopped and that of transition t8 begins. The memoryless property of the exponential distribution makes this counting interruption statistically inessential.

15 A Class of Generalized Stochastic Petri Nets 107 Table III. Legend for the GSPN in Figure 5 Pl P2 P3 P4, P,o, P,6 Ps, P~, P~v ps, p12, P,s P7, P13, PI9 Ps, P14, P20 Ps, P~s, P2~ tl t2, t12, t22 t3, tlz, t23 t4, h4, t24 tb, t~5, t25 te, t,s, t26 t7, t17, t27 ts, t~s, t2s t9, tl9, t2s tlo, t20, t30 tn, t21, t31 active processors available buses processors waiting for accessing common memories idle common memory processor accessing common memory one processor is queued for common memory two processors are queued for common memory three processors are queued for common memory four processors are queued for common memory end of a processor activity a processor seizes a bus and starts an access a processor becomes first in queue for common memory a processor becomes second in queue for common memory a processor becomes third in queue for common memory a processor becomes fourth in queue for common memory end of a memory access; common memory and bus are released end of a memory access; a processor leaves the queue end of a memory access; a processor leaves the queue end of a memory access; a processor leaves the queue end of a memory access; a processor leaves the queue Each time one of the transitions ts to tll completes its counting, a processor returns to execute in its private memory and the bus is used to allow the processor that is next in the queue to take its turn in accessing the common memory. The bus is released only after the last processor in the queue completes its access, that is, after the firing of transition tv. The reachability set of the GSPN comprises 64 tangible states and 22 vanishing states. The GSPN of Figure 5 can be expanded to model a system with a larger number of processors, memories and buses. A change in the number of buses requires only a modification of the initial marking. A change in the number of processors requires a modification of the initial marking and an expansion of the chains of places P5 to ps, p,, to P,5, and P17 to P2, to accommodate as many places as there are processors. Finally, a change in the number of common memories calls for a redrawing of the GSPN, since a new subnet must be added for each new common memory. The detailed system operation description of the GSPN in Figure 5 makes this mod~l" relatively easy to understand at the expense of a nontrivial graphical complexity. A more compact description of the bus and queue management policies can indeed be obtained by avoiding the representation of redundant places and transitions. Figure 6 depicts a subnet that can be used to replace each of the tree subnets of Figure 5. The behavior of the GSPN obtained after having performed this substitution can again be explained by focusing attention on the accessing of the first common memory. When a bus is available and a processor in P3 selects a memory to perform an external access this selection is made according to a uniform distribution. The selection of the first common memory is represented by the firing of the immediate transition t2, so that a token moves from p~ to P4.

16 108 M. Ajmone Marsan, G. Balbo and G. Conte t14 P' t 6~p2 P2 P3 Fig. 6. Compact representation of the subnet in Figure 5. Table IV. Legend for the GSPN in Figure 6 p, a processor wants to access a common memory P7 processors accessing or queued for common memory p,0 the bus selects its next destination t2 selection of a common memory t5 a processor selects an idle common memory te a processor selects a busy common memory tn end of a common memory access t14 the bus is released and the common memory becomes idle t~5 the bus is assigned to the next processor in queue Then only one of the two transitions t5 and t6 can be enabled, according to the following rules. (1) Transition t6 is enabled if another processor is already accessing the same memory and using a bus (m7-1). In this case the bus is released, meaning that the new accessing processor will be waiting for the same memory and for the same bus that is already in use by the processor in PT. (2) Transition t5 is enabled if no processor is accessing the selected memory (m7 = 0). In this case the processor keeps the bus and a token moves from P4 to PT. Since 1/# is the average access time, ~ is the firing rate of tl,. When t,, fires, a token moves to Pl, meaning that the processor is again working on its private memory. At the same time a bus is released (and a token moves to p2) only if no other processor is queued for the same memory (i.e., m7 = 0). These rules are implemented by the pair of immediate transitions t,4 and t,~. Transition tt4 is enabled if m7 = 0. If m7-1, then t,5 is enabled and the bus is not released. The

17 A Class of Generalized Stochastic Petri Nets, 109 (.. PI ). NN ml,~ 1 J P, ms/-~ Fig. 7. Reduced GSPN model of the multiprocessor architecture of Figure 4 (5 processors, 3 memories, 2 buses). The switching distribution defined over t2... ts has the general form Pr(t2) = m4/3, Pr(t3) = m5/3, Pr(t4) = me/3,.pr(t5) = m7/3, Pr(tD = m8/3. reachability set of the GSPN comprises 64 tangible states and 199 vanishing states. Using the subnet of Figure 6 a change in the number of processors and buses induces only a modification of the initial marking, whereas a change in the number of memories requires the redrawing of the GSPN: 6 transitions and 3 places must be added for each new memory. The complexity of the net is thus observed to grow linearly with the number of memories. Since we have assumed that the 3 common memories are statistically equivalent, a different GSPN model of the same system can be built, at the expense of a more careful study of multiprocessor system behavior. Figure 7 depicts this new model. The GSPN now comprises 9 places, 6 timed transitions, and 5 immediate transitions. The 3 memories are represented by three tokens in the same place, P4. Processors executing in their private memories are represented by tokens in place pl. Transition tl models the issuing of an access request. Requests are put in place P3, implicitly assuming that, as in the previous case, no choice among memories has yet been made. Places ps-p9 may contain tokens, each representing a group of processors requesting the same memory. Only one of the processors in the group is served at one time. If the group is made of only one processor the token is in Ps, if there are two processors the token is in Ps, and so on. If five processors are requesting the same memory, one token is in place p9. New requests choose memories, according to a uniform probability distribution. The choice is described in the model by the five immediate transitions t2 to t6, upon which a switching distribution is defined. This GSPN has a lot in common with that of Figure 5 and can indeed be seen as a straightforward reduction of the GSPN obtained by folding onto each other the subnets representing the common memory accesses. Tokens

18 110 M. Ajmone Marsan, G. Balbo and G. Conte Table V. Legend for the GSPN in Figure 7 p~ active processors P2 available buses P3 processors waiting for accessing common memories p4 idle common memory ps processor accessing common memory p6 queues for common memories having length one p7 queues for common memories having length two ps queues for common memories having length three Ps queues for common memories having length four t~ end of a processor activity t2 a processor seizes a bus and starts access t3 a processor becomes first in queue for common memory t4 a processor becomes second in queue for common memory t5 a processor becomes third in queue for common memory ts a processor becomes fourth in queue for common memory t~ end of a memory access; common memory and bus are released t8 end of a memory access; a processor leaves the queue t9 end of a memory access; a processor leaves the queue t~o end of a memory access; a processor leaves the queue tl~ end of a memory access; a processor leaves the queue move along the chain of places P.~-p9, following the same rules discussed during the description of the GSPN in Figure 5. A distinctive characteristic of this new GSPN is that several tokens can now be simultaneously present in the chain of places P~-P9 to represent the queue lengths at the different common memories. Again each token in this chain of places represents a group of processors requesting the same common memory. The random switches in the GSPN comprise transitions in the set {t2... t6 }, and switching distributions whose general form is: Pr(t2) = m4/3, Pr(t3) = mj3, Pr(t~) = m6/3, Pr(ts) = m7/3, Pr(t6) = ms~3. Note that this switching distribution is well defined for all random switches, since in every marking enabling more than one immediate transition the sum m ms equals the number of memories (three in this case). In fact, all marking of this GSPN are such that m4 + + m9 = 3, but whenever m9 > 0 no immediate transition is enabled. The five timed transitions tv to tll represent the common memory accesses. The firing of t7 modifies the marking, moving one token from p~ to Pl and putting one token in both P2 and P4. Upon the firing of ts-th, one token is moved one step to the right in the chain ps-ps, and one token is put in place p~. The policy for the bus arbitration is the same as the one used in the model of Figure 5.

19 A Class of Generalized Stochastic Petri Nets 111,i,1 ~ -u(ms)] Fig. 8. Reduced GSPN model of the multiprocessor architecture of Figure 4 (5 processors, 3 memories, 2 buses). The switching distribution defined over t2 - t3 is Pr(t~) = 1 - i/m, Pr(t3) = 1/m, where m is the number of common memories (m = 3). The unit step function is u(.). J The development of this GSPN requires more modeling effort than is necessary for the GSPN in Figure 5, but results in a much smaller reachability set. Indeed, the number of tangible states is reduced from 64 to 16, and that of vanishing states from 22 to 8. Using this GSPN, the multiprocessor performance analysis thus becomes much simpler. Moreover the structure of this new GSPN is such that multiprocessor systems with a larger number of memories and buses are simply represented by changing the initial marking. An increase in the number of processors requires instead a change in the initial marking and a lengthening of the chain of places Ps-P9. This chain must in fact comprise as many places as there are processors in the system. The GSPN models of Figures 5 and 7 have the distinctive characteristic of being well suited to represent multiprocessor systems with a small number of memories and of processors, respectively. In practice, however, it is much more common that multiprocessor systems comprise a large number of processors and memories, but very few buses. It is thus desirable to develop a GSPN model of the multiprocessor system whose complexity only depends on the number of buses. This can again be done provided that a careful study of the multiprocessor behavior is made. The resulting GSPN model for the 5-processor, 3-memory, and 2-bus multiprocessor system is shown in Figure 8. Only 5 places and 5 transitions (3 timed and 2 immediate) are necessary in this case. Place pl as usual contains tokens representing processors executing in their private memory, andp2 contains tokens representing free buses. Transition tl represents the issuing of access requests, and p3 contains requests that have not yet been served. Tokens in P4 represent processors accessing common memories. Tokens in P5 represent processors requesting the same common memory as the one accessed by the token that has been in p4 longer. The firing of t.~ represents the end of the access to the

20 112 M. Ajmone Marsan, G. Balbo and G. Conte Table VI. Legend for the GSPN in Figure 8 p~ active processors P2 available buses P3 processors waiting for accessing common memories P4 processors accessing common memories ps processors queued for common memories tl end of a processor activity t2 a processor selects an idle common memory t3 a processor selects a busy common memory t4 end of access to a memory with an empty queue t5 end of access to a memory with a nonempty queue memory for which processors in P5 are queued. The firing of t4 represents the end of the access to a memory for which there is no outstanding request. The two immediate transitions, t2 and t3, model the memory choice: the firing of t3 corresponds to choosing the memory that is being accessed by the processor in p4. The choice of any other memory corresponds to the firing of t2. The random switches that must be defined in the GSPN comprise t2 and t3 and the switching distribution: Pr(t2) = 1 - (l/m), Pr(t3) = 1/m, where m is the number of common memories. Again the same switching distribution is valid for all the GSPN markings enabling both t2 and t3. When a token enters P3 three situations are possible: (1) All memories are idle (m4 = 0, m2 = 2). The only enabled transition is t2. (2) One memory is busy (m4 = 1, m2 = 1). Both t2 and t3 are enabled; t3 fires with probability 1/m corresponding to the possibility of choosing the busy memory. (3) Two memories are busy (m4 = 2, m2 = 0). Neither t2 nor t3 is enabled since the system comprises only two buses. The policy for the bus arbitration is the same as in the previous models. The reachability set generated by this GSPN is exactly the same as that obtained with the GSPN in Figure 7. This model can accommodate any number of processors and memories for a fixed number of buses, b, which in our case is equal to 2. To extend the model to the case with b > 2, the chain t3-p~-t5 must be split into b - 1 similar chains, and appropriate connections to the rest of the net must be introduced. The complexity of the net then grows linearly with the number of buses. It must be observed again that the three GSPNs in Figures 5, 7 and 8 model exactly the same system. Let us summarize how the main properties of the system are represented by the different GSPN models: (1) The "explicit" request for a memory (memory choice) is made by the processors only when a bus is available. This is obtained in the three models using a place (p~) in which requests are collected upon generation, before the memory choice. These places are surrounded by a "barrier" of immediate transitions enabled only if a bus is available. ACM Transactions on Computer Systems, Vol. 2; No. 2, May 1984.

21 A Class of Generalized Stochastic Petri Nets 113 (2) The bus arbitration is modeled by the immediate transitions t3-t6, t13-tls, and t23-t2~ in Figure 5, by the chain t3-ts in Figure 7, and by the absence of an output arc from t.~-p3 in Figure 8. A further interesting comment can be made about the way the same queuing policy is modeled by the GSPNs of Figure 7 and 8 with completely different topologies. The chain P~-P9 in Figure 7 maps into places pt and P5 in Figure 8. The number of tokens in p5 (Figure 8) corresponds to the position of the token in chain P6-P9 (Figure 7); note that me + m7 + ms + m9-1 since two buses are available. For example, if m5 = 3 (Figure 8), meaning that a processor is accessing one memory and three other processors are queued for the same memory, in Figure 7 we have ms = NUMERICAL RESULTS As an example of the usefulness of GSPNs in the performance analysis of multiprocessor systems, we present some novel results that refine those that have previously been published in the literature. These new results were obtained from the reduced GSPN models presented in the previous Section. Studies of multiprocessor architectures like the one in Figure 4 in the single bus case have been presented in [2] and [4]. The multiple bus case has been analyzed in [1] and [3]. In those papers, the exact analysis of the multiple bus architecture by means of continuous-time Markov chains was considered too cumbersome, and hence approximate models were introduced. In particular, bounds on the system processing power, PP, defined as the average number of processors executing in their private memory, were shown to be readily obtainable in [1]. An upper bound is obtained by considering only competition for buses, and neglecting competition for common memory modules. This assumption is obviously optimistic (provided there is more than one bus), and leads to reasonably accurate results for multiprocessor systems comprising a small number of buses and a much larger number of common memories. Indeed, in this case, whenever a bus is available, only a few common memories are actually being accessed (at most b - 1), and there is a high probability of selecting a free one. Note that the multiprocessor performance tends to this upper bound when the number of common memories increases. The evaluation of the upper bound is straightforward, since it entails the steady state solution of either an M/M/b queue with finite population (p customers), or, equivalently, the central server queuing network shown in Figure 9. Closed form solutions can be found in most queuing theory books, and a short derivation is presented in [1]. The derivation of a lower bound on the processing power of these multiprocessor systems requires the use of slightly more complex arguments. One way of obtaining a lower bound on the system performance is to introduce a pessimistic assumption that increases the contention for shared resources. To make the computation of this lower bound simple, this pessimistic assumption should also have the feature of drastically simplifying the model. The pessimistic assumption used in [1] to obtain a lower bound on processing power is to consider a

22 114 M. Ajmone Marsan, G. Balbo and G. Conte 1 Fig. 9. Upper bound queuing network. D i I P F Fig. 10. Lower bound queuing network. multiprocessor system in which the number of common memories is reduced to the number of buses. Since the number of system resources is decreased, the processing power must be lower than in the original model. Indeed, consider the case in which only b - 1 resources are being accessed, so that one bus is still available. In the original model, a processor issuing a request references a free resource with probability (m - b + 1)/m. In the simplified model, the probability of referencing the only free resource is l/b, which is smaller than in the previous case, provided that m > b, and that b > 1. The lower bound model is obviously exact when the number of common memories in the system is equal to the number of buses. In this case however, the multiprocessor system structure can be viewed as a crossbar, rather than as a bus architecture. If the number of common memories is larger than the number of buses (and there is more than one bus), the lower bound is only an approximation of the system performance. The accuracy of the approximation decreases for an increasing number of common memories. The evaluation of the lower bound entails the solution of the closed queuing network of Figure 10, that comprises one infinite server station with an exponential service time, and b FCFS single server stations with exponential service