Parallel Architectures Group Grupo de Arquitecturas Paralelas (GAP)

Transcription

1 Handling in Interconnection Deadlock Networks Parallel Architectures Group Switching Techniques, Adaptive Routing and Jose Duato de Ingeniera de Sistemas, Computadores y Automatica Dept. Politecnica de Valencia, Spain Universidad 1

2 Adaptive Routing and Deadlock Handling in Interconnection Networks Jose Duato de Ingeniera de Sistemas, Computadores y Automatica Dept. Politecnica de Valencia, Spain Universidad 1

3 Outline Introduction Switching techniques Optimized switching techniques Deadlock handling Theory of deadlock avoidance Design methodologies Application to deadlock recovery Application to networks of workstations Performance evaluation 2

4 Outline Introduction Deadlock handling Theory of deadlock avoidance Design methodologies Application to deadlock recovery Application to networks of workstations Performance evaluation 2

5 Introduction (From W. J. Dally) performance of most digital systems today is limited by their The or interconnection, not by their logic or memory communication of the power is used to drive wires and most of the clock cycle Most spent on wire delay, not gate delay is technology improves, pin density and wiring density are scaling at As slower rate than the components themselves. Also, the frequency a communication between components is lagging far beyond the of rates of modern processors clock factors combine to make interconnection the key factor in the These of future digital systems success 3

6 Introduction (From W. J. Dally) designers strive to make more ecient use of scarce interconnection As bandwidth, interconnection networks are emerging as a nearly solution to the system-level communication problems for universal digital systems modern developed for the demanding communication requirements Originally multicomputers, interconnection networks are beginning to re- of place buses as the standard system-level interconnection networks are also replacing dedicated wiring in Interconnection systems as designers discover that routing packets special-purpose is both faster and more economical than routing wires 4

7 Wide area networks Parallel Architectures Group Introduction networks are currently being used for many dierent Interconnection ranging from internal buses in VLSI circuits to wide applications, area computer networks. These applications include: System area networks Telephone switches Internal networks for ATM switches Processor/memory interconnects for vector supercomputers Interconnects for multicomputers Interconnects for distributed shared-memory multiprocessors Clusters of workstations Local area networks Metropolitan area networks } Computer networks 5

8 computers should be designed using commodity components Parallel be cost-eective to commodity communication subsystems have been Unfortunately, to meet a dierent set of requirements, i.e., those arising designed high performance interconnection networks becomes a Designing issue to exploit the performance of parallel computers critical several high performance switches have been developed Recently, build inexpensive parallel computers by connecting cost-eective to Parallel Architectures Group Introduction in computer networks Most manufacturers designed custom interconnection networks computers through those switches 6

9 Deterministic routing, adaptive routing Packet switching, circuit switching, wormhole, virtual cut-through Parallel Architectures Group Main design parameters Denes how the nodes are interconnected by channels Topology: Direct networks, switch-based networks algorithm: Determines the path selected by amessage to Routing its destination reach technique: Determines how and when buers are Switching and switches are congured reserved 7

10 8 Interconnection Networks Shared-Medium Networks Local Area Networks Contention Bus (Ethernet) Token Bus (Arcnet) Token Ring (FDDI Ring, IBM Token Ring) Backplane Bus (Sun Gigaplane, DEC AlphaServer8X00, SGI PowerPath-2) Direct Networks (Router-Based Networks) Strictly Orthogonal Topologies Mesh 2-D Mesh (Intel Paragon) 3-D Mesh (MIT J-Machine) Torus (k-ary n-cube) 1-D Unidirectional Torus or Ring (KSR first-level ring) 2-D Bidirectional Torus (Intel/CMU iwarp) 3-D Bidirectional Torus (Cray T3D, Cray T3E) Hypercube (Intel ipsc, ncube) Other Topologies: Trees, Cube-Connected Cycles, de Bruijn, Star Graphs, etc. Indirect Networks (Switch-Based Networks) Hybrid Networks Regular Topologies Crossbar (Cray X/Y-MP, DEC GIGAswitch, Myrinet) Multistage Interconnection Networks Blocking Networks Unidirectional MIN (NEC Cenju-3, IBM RP3) Bidirectional MIN (IBM SP, TMC CM-5) Nonblocking Networks: Clos Network Irregular Topologies (DEC Autonet, Myrinet, ServerNet) Multiple-Backplane Buses (Sun XDBus) Hierarchical Networks (Bridged LANs, KSR) Cluster-Based Networks (Stanford DASH, HP/Convex Exemplar) Other Hypergraph Topologies: Hyperbuses, Hypermeshes, etc. Parallel Architectures Group

11 Direct networks (a) 2-ary 4-cube (hypercube) (b) 3-ary 2-cube (c) 3-ary 3D-mesh 9

12 Multistage interconnection networks Multistage butterfly network Omega network 10

13 Switch-based irregular topologies Bidirectional Links Switch Processing Elements 6 Switch-Based Network Graph Representation 11

14 Generalized MIN model N M P o r t s P o r t s C 0 G 0 C 1 G 1 G g 1 C g 12

15 manufacturers developed switches that are suitable to implement Some either direct or indirect networks (Inmos C104, SGI SPIDER) can view networks using point-to-point links as a set of We switches, each one connected to zero, one, or more interconnected Direct networks correspond to the case where every switch is to a single node connected Crossbar networks correspond to the case where there is a single connected to all the nodes switch Multistage interconnection networks correspond to the case switches are arranged into several stages and the switches where Parallel Architectures Group Unied View nodes: in intermediate stages are not connected to any processor 13

16 Router organization Injection Channel LC LC Ejection Channel LC LC Input Channels LC LC Switch LC LC Output Channels LC Routing & Arbitration LC 14

17 Switching Switching: Determines how and when buers are reserved and switches are congured control: Synchronization protocol for transmitting and Flow a unit of information receiving of ow control: Portion of the message whose transfer must Unit synchronized be control occurs at two levels: message ow control and physical Flow ow control channel 15

18 Time-space diagram (packet switching) Packet switching and circuit switching Channel Time Time-space diagram (circuit switching) Channel Time 16

19 Virtual cut-through and wormhole switching T D D D D D D D D D D D D D D H Time-space diagram Channel Time 17

20 Virtual channels Virtual channel controller From switch Channel multiplexor Physical channel Channel demultiplexor To switch Flit buffers Flit buffers 18

21 Performance of switching techniques Packet switching is well suited for very short messages Circuit switching is well suited for very long messages cut-through switching is well suited for messages of any Virtual but requires splitting messages into xed-size packets length switching is well suited for messages of any length but Wormhole at moderate loads. Virtual channels alleviate this situation saturates switching has been preferred for electronic routers because Wormhole buers can be small and the resulting circuits are compact and fast 19

22 Optimized switching techniques from real applications may be bimodal and may vary over Trac time Wormhole switching can be used for short messages Circuit switching can be used for very long messages set-up can be overlapped with useful computation and/or Path can be reused circuits circuits do not need buers at intermediate routers and can Physical made much faster than conventional links either by using wave be pipelining or optical technology 20

23 Optimized router organization 21 From/to Local Processor Input Channels Pipelined Input Channels Sync Sync Sync Sync Switch S Switch S Control Channels k 1 Switch S 0 Wormhole Control Unit PCS Control Unit mux mux mux Pipelined Output Channels Output Channels Parallel Architectures Group

24 Performance for multimedia applications CS 28+4 WSNR WH Average Latency (cycles) short 10% messages (16 its) long 90% messages (1024 its) Traffic ( CLK x 2) (flits/node/cycle) 22

25 Performance for multimedia applications Average Latency (cycles) CS 28+4 WSNR WH short 10% messages (16 its) long 90% messages (1024 its) Traffic ( CLK x 3) (flits/node/cycle) 23

26 Performance for multimedia applications Average Latency (cycles) CS 28+4 WSNR WH short 10% messages (16 its) long 90% messages (1024 its) Traffic ( CLK x 4) (flits/node/cycle) 24

27 Performance for multimedia applications Average Latency (cycles) WS WH WH 2 VC WH 3 VC short 10% messages (16 its) long 90% messages (1024 its) Only long Long messages traffic (1024 flits long, 90%) messages shown are 25

28 Performance for multimedia applications Average Latency (cycles) WS WH WH 2 VC WH 3 VC short 10% messages (16 its) long 90% messages (1024 its) Only short Short messages traffic (16 flits long, 10%) messages shown are 26

29 Performance for multimedia applications Average Latency (cycles) WS WH WH 2 VC WH 3 VC short 10% messages (16 its) long 90% messages (1024 its) Only short e-4 2.0e-4 3.0e-4 4.0e-4 Short messages traffic (16 flits long, 10%) messages shown are 27

30 Performance for multimedia applications slots 16 slots 8 slots 4 slots 2 slots 1 slot short 10% messages Average Latency (cycles) (16 its) long 90% messages (1024 its) Traffic for 256 Gbps (10% 16 flits, 90% 1024 flits) Gbps 256 band- link width 28

31 Performance for multimedia applications slots 16 slots 8 slots 4 slots 2 slots 1 slot short 40% messages (16 its) Average Latency long 60% messages (1024 its) Traffic for 256 Gbps (40% 16 flits, 60% 1024 flits) Gbps 256 band- link width 29

32 Routing Algorithms Number of Destinations Unicast Routing Multicast Routing Routing Decisions Centralized Routing Source Routing Distributed Routing Multiphase Routing Implementation Table Lookup Finite-State Machine Adaptivity Deterministic Routing Adaptive Routing Progressiveness Progressive Backtracking Minimality Profitable Misrouting Number of Paths Complete Partial 30

33 Undeliverable Packets Situations that may prevent packet delivery Deadlock Prevention Avoidance Recovery Livelock Starvation Minimal Paths Restricted Nonminimal Paths Probabilistic Avoidance Resource Assignment Scheme 31

34 Deadlock handling Deadlock prevention: Backtracking Deadlock avoidance: Acyclic graph, acyclic subgraph Regressive deadlock recovery: Message removal, message abortion Progressive deadlock recovery: Disha Main goal Design of ecient deadlock-free fully adaptive routing algorithms 32

35 conguration Deadlocked N N N1 wait for resources Messages held by other messages in a cyclic way Removing cyclic dependencies ) will avoid deadlock N2 33

36 Allowing cyclic dependencies for the unidirectional ring: c Ai channels can be used to Example messages to all the destinations. c Hi channels can only be forward n n c A3 c c c c H0 A0 A2 H2 c A1 n n 2 c H1 exist cyclic dependencies between There Ai channels c c Hi However, dependencies channels have no cyclic is no deadlock because messages There for resources can always escape waiting by using c Hi channels used if the destination is higher than the current node. 34

37 Theory of deadlock avoidance (informal) Interconnection network 35

38 Adaptive routing function and selection function n c n c n c n d n d n d Routing Function Selection Function 36

39 routing function will be referred to as routing subfunction when The to escape channels restricted Parallel Architectures Group Routing subfunction channels can be split into two subsets: adaptive and escape Network channels 37

40 Approach to avoid deadlock adaptive routing function may allow cyclic dependencies between An as long as: channels There exist a subset of channels (escape channels) that have no dependencies between them cyclic It is possible to establish a path from the current node to the node using only escape channels destination For wormhole switching, when a message reserves an escape and then an adaptive channel, it must be able to select channel escape channel at the current node, i.e., escape channels an have no cyclic dependencies indirectly through adaptive should channels 38

41 ) There is a deadlock Parallel Architectures Group Deadlock produced by indirect dependencies set of messages are cyclically A for channels occupied by waiting other messages in the set messages are able to use Some channels but reach an- escape cycle. Messages using escapother channels are cyclically wait- indirectly through adaptive ing channels 39

42 Extend the network topology and the routing function Guarantee the absence of deadlocks Parallel Architectures Group Design methodology Based on the extension of other routing functions Allows the use of all the alternative minimal paths Does not increase the number of physical channels Provides a way to: 40

43 Given an interconnection network I1, dene a minimal path deadlock-free routing function R1 connected to a minimal path or, alternatively, the channels supplied by R1 Verify that the extended channel dependency graph for R1 Parallel Architectures Group Design methodology Steps: Split each physical channel into a set of additional virtual channels. new routing function can use any of the new channels belonging The is If it is, the routing algorithm is valid. Otherwise, it must acyclic. discarded. This step is not required for store-and-forward and be virtual cut-through 41

44 Split each physical channel c i into k virtual channels Step2: i;1 ;a i;2 ;:::;a i;k,1 ;b i a Parallel Architectures Group Design example Routing algorithm for n-dimensional meshes Basic algorithm: Dimension order routing algorithm: Route over any minimal path using any of the a New Alternatively, route over the lowest useful dimension using channels. the corresponding b channel MIT Reliable Router uses two virtual channels for fully adaptive The routing and two virtual channels for dimension-order routing minimal in the absence of faults (on a 2-D mesh) 42

45 Example routing paths for 2-D meshes Source node Destination node Channels supplied by R

46 Extended channel dependency graph for R1 Parallel Architectures Group b10 b01 b12 b21 b03 b14 b25 b30 b52 b34 b45 b43 b54 b36 b58 b63 b74 b85 b67 b76 b87 b78 44

47 Average Latency (cycles) Performance evaluation for the 2-D mesh Deterministic (1 vc) Deterministic (2 vc) Adaptive (2 vc) size: Network processors. 256 length: Message its Random trac Normalized Accepted Traffic 45

48 Average Latency (cycles) Performance evaluation for the 3-D mesh Deterministic (1 vc) Deterministic (2 vc) Adaptive (2 vc) 0.52 size: Network processors. 512 length: Message its. 16 Random trac Normalized Accepted Traffic 46

49 Average Latency (cycles) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 256 length: Message its. 16 Performance evaluation for the 2-D torus Random trac Normalized Accepted Traffic 47

50 Average Latency (cycles) Performance evaluation for the 3-D torus Deterministic (2 vc) Part-Adaptive (2 vc) Part-Adaptive (3 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its Random trac Normalized Accepted Traffic 48

51 Performance evaluation for the 3-D torus (II) 80 Average Latency (cycles) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its. 16 Local trac Normalized Accepted Traffic 49

52 Performance evaluation for the 3-D torus (III) 100 Average Latency (cycles) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its. 16 Bit-reversal pattern trac Normalized Accepted Traffic 50

53 Accurate performance evaluation for the 3-D torus Average Latency (ns) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its. 16 Random trac Traffic (flits/node/us) 51

54 Accurate performance evaluation for the 3-D torus (II) Average Latency (ns) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its. 16 Local trac Traffic (flits/node/us) 52

55 Accurate performance evaluation for the 3-D torus (III) 700 Average Latency (ns) Deterministic (2 vc) Part-Adaptive (2 vc) Adaptive (3 vc) size: Network processors. 512 length: Message its. 16 Bit-reversal pattern trac Traffic (flits/node/us) 53

56 Application to deadlock recovery resources (channels or buers) are split into two classes: Routing and escape adaptive Adaptive resources can be freely used by all the packets a packet is waiting for longer than a timeout, it moves to an When resource escape a packet uses an escape resource, it cannot use an adaptive Once again resource routing scheme eliminates all the indirect dependencies between This and escape resources adaptive 54

57 Injection Channel LC LC Ejection Channel Router organization for Disha LC LC Input Channels LC LC Switch LC LC Output Channels LC LC Routing and Arbitration Deadlock Buffer 55

58 Routing on edge and deadlock buers buers can only be used in increasing Deadlock order label a deadlock is detected, the packet header When be routed to the deadlock buer can Edge buers allow fully adaptive minimal routing channels are dened so that the routing Escape is able to deliver messages for any subfunction destination (including deadlock buers) 56

59 Extended channel dependency graph for edge buers n0 c10 n1 c21 n2 c50 n5 c41 n4 c32 n3 c65 c74 c83 n6 n7 n8 57

60 180 Performance evaluation Average Latency (Cycles) o Avoidance Det (2 VC) x Recovery Det (2 VC) + Avoidance Adap (3 VC) * Recovery Adap (3 VC) Normalized Accepted Traffic 58

61 Injection limitation Prevents performance degradation at saturation Reduces the frequency of deadlock occurrence to negligible values RESERVE RELEASE BUSY OUTPUT CHANNELS COUNTER THRESHOLD COMPARATOR INJECTION PERMITTED 59

62 Improved injection limitation mechanism PHYSICAL CHANNELS Vn-1 V1 V0 0 RESERVE RELEASE MESSAGE NUM BUSY OUTPUT CHANNELS m-1 BIT =1 counter COUNTER TRANSLATION TABLE COMPARATOR INJECTION PERMITTED Bitwise OR 60

63 Improved deadlock detection mechanism Counter Input Channels Switch I Threshold Output Channels Counter I Thresho ld 61

64 Wiring exibility. Scalability. Incremental expansion capability. Parallel Architectures Group Application to networks of workstations of workstations are emerging as a cost-eective alternative Networks parallel computers. to interconnects like Autonet, Myrinet and ServerNet Switch-based been proposed to build networks of workstations with irregular have topology. The irregularity provides: 62

65 The irregularity makes deadlock avoidance and routing Drawback: complicated. quite ) Many messages are routed following non-minimal paths.! Higher message latency! Waste of resources! Lower throughput! Reduces contention by increasing routing adaptivity! Allows more messages to follow minimal paths Parallel Architectures Group Simplest solution: Avoid deadlock by eliminating all the cyclic dependencies between channels Alternative solution: Allow cyclic dependencies between channels 63

66 Switch-based networks with irregular topologies Bidirectional Links Switch Processing Elements 6 Switch-Based Network Graph Representation 64

67 Deadlock-free routing scheme (up/down routing). Provides partially adaptive communication between nodes. Distributed. Implemented using table-lookup. Parallel Architectures Group The Autonet routing algorithm General characteristics: 65

68 Each cycle has at least one link in the \up" direction and one in the \down" direction. link Cyclic dependencies are avoided: messages cannot cross a link the \up" direction after one in the \down" direction. in Parallel Architectures Group "up" direction 0 The up/down routing algorithm is based on an assignment Routing direction to the operational links. of Routing rule: a legal route must zero or more links in the traverse direction followed by zero or \up" more links in the \down" direction. 66

69 From 7 to 0: OK From 2 to 5: lack of adaptivity From 4 to 1: non-minimal routing basic routing rule prevents from using minimal routing and The in most cases because of \down" to \up" conicts. adaptivity Parallel Architectures Group "up" direction 0 Routing eciency Probability of non-minimal routing increases with network size. 67

70 A design methodology for adaptive routing algorithms channels physical or split into duplicated interconnection network + deadlock-free new methodology virtual channels two and new) (original + routing extended routing function ) function 68

71 Extended routing function Newly injected messages can use the new channels without any For performance reasons, only minimal paths are restriction. allowed Original channels are used exactly in the same way as in the routing function original Once a message reserves one of the original channels, it cannot any of the new channels again use When the routing table provides both kinds of channels, give to new channels preference The extended routing function is deadlock-free 69

72 At intermediate switches, a higher priority is assigned to the { channels belonging to minimal paths new If all the new channels are busy, then an original channel { to a minimal path (if any) is selected belonging If none exists, then the one that provides the shortest path is { (this ensures deadlock-freedom) used Once a message reserves an original channel, it can no longer a new one reserve Parallel Architectures Group Improving the eciency of the methodology Idea: Focus on minimal routing, even if adaptivity is reduced Restrict the transition from new channels to original channels Improved adaptive routing function: Newly injected messages can only use new channels { 70

73 Basic up/down routing scheme (UD). Parallel Architectures Group Performance evaluation Evaluation of four routing schemes: Up/down routing scheme using two virtual channels per physical (UD-2VC). channel Adaptive routing scheme using two virtual channels per physical (A-2VC). channel Improved adaptive routing scheme using two virtual channels per channel (MA-2VC). physical Performance evaluation carried out by simulation. 71

74 Topology generated randomly (8-port switches) 4 nodes (processors) connected to each switch Two adjacent switches are connected by a single link Message destination is randomly chosen among nodes Parallel Architectures Group Network model: One routing control unit per switch (assigned in a round-robin fashion) It takes one clock cycle to compute the routing algorithm, to one it from an input buer to an output buer, or to transfer transfer one it across a physical channel 72

75 Simulation results (I) Average Latency (Cycles) UD UD-2VC A-2VC MA-2VC size: 16 Network switches. length: Message its Traffic (Flits/Cycle/Node) 73

76 Simulation results (II) Average Latency (Cycles) UD UD-2VC A-2VC MA-2VC size: 32 Network switches. length: Message its Traffic (Flits/Cycle/Node) 74

77 Simulation results (III) Average Latency (Cycles) UD UD-2VC A-2VC MA-2VC size: 64 Network switches. length: Message its Traffic (Flits/Cycle/Node) 75

78 Average Latency (Cycles) 200 Simulation results (IV) UD UD-2VC A-2VC MA-2VC size: 64 Network switches. length: Message its Traffic (Flits/Cycle/Node) 76

79 Average Latency (Cycles) 900 Simulation results (V) UD UD-2VC A-2VC MA-2VC size: 64 Network switches. length: Message its Traffic (Flits/Cycle/Node) 77

80 Amount of messages Simulation results for application traces Messages Traces from Barnes-Hut on executed processors 0 1e+07 2e+07 3e+07 4e+07 5e+07 6e+07 7e+07 8e+07 Time 78

81 Simulation results for application traces MA-2VC UD-2VC UD Latency (Cycles) e+07 2e+07 3e+07 4e+07 5e+07 6e+07 7e+07 8e+07 Time (Cycles) 79

82 Zoom of the rst peak MA-2VC UD-2VC UD Latency (Cycles) e e e e+07 2e+07 Time (Cycles) 80

83 Zoom of the second peak MA-2VC UD-2VC UD Latency (Cycles) e e e e e+07 Time (Cycles) 81

84 Zoom of the third peak MA-2VC UD-2VC UD Latency (Cycles) e e e e e+07 Time (Cycles) 82

85 switching techniques may considerably increase performance Hybrid using the appropriate switching technique for each message class by switching can take advantage of wave pipelining and optical Circuit to increase link bandwidth technology deadlock avoidance and recovery schemes allow the design Flexible more ecient routing algorithms of routing algorithms have been implemented in the MIT Reliable These and the Cray T3E Router routing and virtual channels are especially interesting when Adaptive produce bursty trac that saturates the network during applications routing and virtual channels must be implemented eciently Adaptive to minimize the increment in clock cycle time Parallel Architectures Group Final Remarks some time intervals (usually prior to synchronization points) 83

86 Final Remarks deadlock avoidance and recovery schemes allow the design Flexible more ecient routing algorithms of routing algorithms have been implemented in the MIT Reliable These and the Cray T3E Router routing and virtual channels are especially interesting when Adaptive produce bursty trac that saturates the network during applications some time intervals (usually prior to synchronization points) routing and virtual channels must be implemented eciently Adaptive to minimize the increment in clock cycle time 83