Technology-Driven, Highly-Scalable Dragonfly Topology

Size: px

Start display at page:

Download "Technology-Driven, Highly-Scalable Dragonfly Topology"

Leonard Norman
10 years ago
Views:

1 Technology-Driven, Highly-Scalable Dragonfly Topology By William J. Dally et al ACAL Group Seminar Raj Parihar

2 Motivation Objective: In interconnect network design Minimize (latency, diameter, cost) 2-D Mesh (Low-Radix Routers) 2-D Mesh (High-Radix Routers) One Solution Use high-radix routers 3/12/2012 ACAL Group Seminar Raj Parihar 2

Routers) 2-D Mesh (High-Radix Routers) One Solution Use

3 Outline High-Radix Routers Global Cables: Cost Models Dragonfly Topology Routing in Dragonfly Cost Comparison Related Work Conclusion 3/12/2012 ACAL Group Seminar Raj Parihar 3

4 Low-Radix vs High-Radix Router Driving forces Technology evolution (Active Optical Cables) Increased pin-bandwidth Idea is to distribute the increased pin-bw to more number of ports Advantage: Reduced number of hops Disadvantage: Network cables are longer R R Low-Radix High-Radix 3/12/2012 ACAL Group Seminar Raj Parihar 4

pin-bw to more number of ports Advantage: Reduced number of hops Disadvantage:

5 Global Cables: Cost Model Comparison Break-even analysis If length of link is less than 10 m use electrical cables else use optical cables Active* optical cables can benefit in two ways Use of longer global cables Use of less number of cables 3/12/2012 ACAL Group Seminar Raj Parihar 5

Active* optical cables can benefit in two ways Use of longer global

6 High-Radix Routers Pros Cons Reduced network diameters and latency Reduced network cost by minimizing the number of global (read as expensive) cables Increases the length of global cables Increases the complexity of routers Benefits from use of emerging active optical cables encourage to use high-radix routers 3/12/2012 ACAL Group Seminar Raj Parihar 6

global cables Increases the complexity of routers Benefits from use of emerging active

7 High-Radix Routers As technology advances pin-bw increases Idea is to use more number of narrow ports As oppose to wide less number of ports Benefits of high-radix routers Only 1 global hops for each packets 3/12/2012 ACAL Group Seminar Raj Parihar 7

less number of ports Benefits of high-radix routers Only 1

8 Dragonfly Topology: Evolution 2-D Mesh (with High-Radix Routers) Island (Virtual Router) Dragonfly Global channel (Optical Links) 3/12/2012 ACAL Group Seminar Raj Parihar 8

9 Dragonfly: Evolution (2) NWK Island (Virtual Router) Dragonfly Global channel (Optical Links) 3/12/2012 ACAL Group Seminar Raj Parihar 9

10 Dragonfly based Network K = P + A -1 + H K = A (P + H) N = A*P (A*H + 1) N Total number of network Terminals P Number of terminal to each router A number of routers in each group K Radix of the routers K Effective radix of groups (VR) G Number of groups in system H number of global channels 3/12/2012 ACAL Group Seminar Raj Parihar 10

in each group K Radix of the routers K Effective radix of groups (VR) G Number of

11 Topology Description Hierarchical network with three levels Routers Groups and Systems Group radix k >> k Ultimate desire is to achieve the global diameter of ONE Example: N = 72 Global channels Local channels 3/12/2012 ACAL Group Seminar Raj Parihar 11

is to achieve the global diameter of ONE Example: N = 72 Global

12 Scalability Network size vs Router radix Maximum 3 hops: One Global, Two Local 3/12/2012 ACAL Group Seminar Raj Parihar 12

13 Quick Fact Why it is called Dragonfly? A Dragonfly has narrow wings and wide body Similarly, this too has large number of groups whereas channels connecting the groups are less and narrow. 3/12/2012 ACAL Group Seminar Raj Parihar 13

this too has large number of groups whereas channels

14 Routing in Dragonfly Minimal routing works well in load-balanced traffic For adversarial traffic, Valiant s algorithm is applied at system level each packet is routed to random group Randomized non-minimal routing at most two global channels 3/12/2012 ACAL Group Seminar Raj Parihar 14

system level each packet is routed to random group Randomized

15 Routing: Evaluation Four routing algorithms (MIN, VAL, UGAL L, G) Uniform random traffic Adversarial traffic pattern 3/12/2012 ACAL Group Seminar Raj Parihar 15

16 Indirect Adoptive Routing Adoptive routing is challenging Routers may not be directly connected to global channels Each router must pick a global channel using only local information Local information depends only indirectly on the state of the global channel Challenge is How to implement the indirect adoptive routing? 3/12/2012 ACAL Group Seminar Raj Parihar 16

information Local information depends only indirectly on the state of the global channel

17 Cost Comparison For network size N > 16K nodes 20% compare to flattened butterfly 52% compare to folded Clos network 3/12/2012 ACAL Group Seminar Raj Parihar 17

18 Related Work SOENet Scalable Opto-Electronic Network Use of sub-networks (islands) Connect islands using long optical cables Many hierarchical topologies has been presented Hierarchical topology using hyper-cubes SGI origin: A ccnuma Highly Scalable Server What is new in Dragonfly? High-Radix virtual routers 3/12/2012 ACAL Group Seminar Raj Parihar 18

presented Hierarchical topology using hyper-cubes SGI origin: A ccnuma Highly Scalable

19 Conclusion Dragonfly topology Group of routers as high-radix virtual router High-Radix routers are effective Reduce the network diameter, latency, cost For global cables it is recommended to use emerging optical cables Reduces overall cost of network 3/12/2012 ACAL Group Seminar Raj Parihar 19

latency, cost For global cables it is recommended to use emerging optical

From Hypercubes to Dragonflies a short history of interconnect

From Hypercubes to Dragonflies a short history of interconnect William J. Dally Computer Science Department Stanford University IAA Workshop July 21, 2008 IAA: # Outline The low-radix era High-radix routers