Technology-Driven, Highly-Scalable Dragonfly Topology By William J. Dally et al ACAL Group Seminar Raj Parihar parihar@ece.rochester.edu
Motivation Objective: In interconnect network design Minimize (latency, diameter, cost) 2-D Mesh (Low-Radix Routers) 2-D Mesh (High-Radix Routers) One Solution Use high-radix routers 3/12/2012 ACAL Group Seminar Raj Parihar 2
Outline High-Radix Routers Global Cables: Cost Models Dragonfly Topology Routing in Dragonfly Cost Comparison Related Work Conclusion 3/12/2012 ACAL Group Seminar Raj Parihar 3
Low-Radix vs High-Radix Router Driving forces Technology evolution (Active Optical Cables) Increased pin-bandwidth Idea is to distribute the increased pin-bw to more number of ports Advantage: Reduced number of hops Disadvantage: Network cables are longer R R Low-Radix High-Radix 3/12/2012 ACAL Group Seminar Raj Parihar 4
Global Cables: Cost Model Comparison Break-even analysis If length of link is less than 10 m use electrical cables else use optical cables Active* optical cables can benefit in two ways Use of longer global cables Use of less number of cables 3/12/2012 ACAL Group Seminar Raj Parihar 5
High-Radix Routers Pros Cons Reduced network diameters and latency Reduced network cost by minimizing the number of global (read as expensive) cables Increases the length of global cables Increases the complexity of routers Benefits from use of emerging active optical cables encourage to use high-radix routers 3/12/2012 ACAL Group Seminar Raj Parihar 6
High-Radix Routers As technology advances pin-bw increases Idea is to use more number of narrow ports As oppose to wide less number of ports Benefits of high-radix routers Only 1 global hops for each packets 3/12/2012 ACAL Group Seminar Raj Parihar 7
Dragonfly Topology: Evolution 2-D Mesh (with High-Radix Routers) Island (Virtual Router) Dragonfly Global channel (Optical Links) 3/12/2012 ACAL Group Seminar Raj Parihar 8
Dragonfly: Evolution (2) NWK Island (Virtual Router) Dragonfly Global channel (Optical Links) 3/12/2012 ACAL Group Seminar Raj Parihar 9
Dragonfly based Network K = P + A -1 + H K = A (P + H) N = A*P (A*H + 1) N Total number of network Terminals P Number of terminal to each router A number of routers in each group K Radix of the routers K Effective radix of groups (VR) G Number of groups in system H number of global channels 3/12/2012 ACAL Group Seminar Raj Parihar 10
Topology Description Hierarchical network with three levels Routers Groups and Systems Group radix k >> k Ultimate desire is to achieve the global diameter of ONE Example: N = 72 Global channels Local channels 3/12/2012 ACAL Group Seminar Raj Parihar 11
Scalability Network size vs Router radix Maximum 3 hops: One Global, Two Local 3/12/2012 ACAL Group Seminar Raj Parihar 12
Quick Fact Why it is called Dragonfly? A Dragonfly has narrow wings and wide body Similarly, this too has large number of groups whereas channels connecting the groups are less and narrow. 3/12/2012 ACAL Group Seminar Raj Parihar 13
Routing in Dragonfly Minimal routing works well in load-balanced traffic For adversarial traffic, Valiant s algorithm is applied at system level each packet is routed to random group Randomized non-minimal routing at most two global channels 3/12/2012 ACAL Group Seminar Raj Parihar 14
Routing: Evaluation Four routing algorithms (MIN, VAL, UGAL L, G) Uniform random traffic Adversarial traffic pattern 3/12/2012 ACAL Group Seminar Raj Parihar 15
Indirect Adoptive Routing Adoptive routing is challenging Routers may not be directly connected to global channels Each router must pick a global channel using only local information Local information depends only indirectly on the state of the global channel Challenge is How to implement the indirect adoptive routing? 3/12/2012 ACAL Group Seminar Raj Parihar 16
Cost Comparison For network size N > 16K nodes 20% compare to flattened butterfly 52% compare to folded Clos network 3/12/2012 ACAL Group Seminar Raj Parihar 17
Related Work SOENet Scalable Opto-Electronic Network Use of sub-networks (islands) Connect islands using long optical cables Many hierarchical topologies has been presented Hierarchical topology using hyper-cubes SGI origin: A ccnuma Highly Scalable Server What is new in Dragonfly? High-Radix virtual routers 3/12/2012 ACAL Group Seminar Raj Parihar 18
Conclusion Dragonfly topology Group of routers as high-radix virtual router High-Radix routers are effective Reduce the network diameter, latency, cost For global cables it is recommended to use emerging optical cables Reduces overall cost of network 3/12/2012 ACAL Group Seminar Raj Parihar 19