Scalable Mul*- Class Traffic Management in Data Center Backbone Networks

Transcription

1 Scalable Mul*- Class Traffic Management in Data Center Backbone Networks Amitabha Ghosh (UtopiaCompression) Sangtae Ha (Princeton) Edward Crabbe (Google) Jennifer Rexford (Princeton)

2 Outline Mo*va*on Contribu*ons Model and Formula*on Scalable Designs Performance Evalua*on Conclusions

3 Mo*va*ons q Mul*ple interconnected data centers (DCs) with mul*ple paths between them q DCs, traffic sources, and backbone owned by the same OSP, e.g., Google, Yahoo, MicrosoR q Traffic with different performance requirements q Different business importance Backbone Data center & host

4 Contribu*ons Controlling the three knobs q Sending rates of hosts q Weights on link schedulers Joint op*miza*on of rate control, rou*ng, and link scheduling q SpliYng of traffic across paths Data center backbone Source DC & host Edge router Des*na*on host & DC

5 Contribu*ons TRUMP (CoNEXT 07) DaVinci (CoNEXT 08) Fully- centralized Not scalable Semi- centralized: Our work Modular, scalable with low message- passing and fast convergence Fully- distributed Scaling issues due to message passing, slow convergence q Computa*on is distributed across mul*ple *ers using a few controllers q Result is provably op*mal using op*miza*on decomposi*on q Semi- centralized solu*ons viable and, in fact, preferred in prac*ce, e.g., Google s B4 globally- deployed sorware defined private WAN (SIGCOMM 13)

6 Model and Formula*on Traffic Model q Performance requirements Set of traffic classes K = {k} q Mul*ple flows per class Flow: traffic between a source- des*na*on pair s 2 F k q Business importance flow weight w k s U*lity Func*on of a Class q All flows in the same class have the same u*lity func*on U k ( ) q For simplicity, assume only throughput and delay sensi*ve traffic f k ( ) g k ( )

7 Model and Formula*on Network Model q Set of unidirec*onal links Capacity c l Propaga*on delay p l L = {l} q Set of paths P = {p} q Rou*ng matrix q One queue per class A =[A lp ] Topology matrix smaller R = Rsp k Path rou*ng matrix larger q Mul*- path rou*ng Path rate of flow s of class k over path p z k sp

8 Model and Formula*on U*lity of Flow s of Class k U k s = w k s Coefficients to model different degrees of sensi*vity to throughput and delay apple a k f k (x k s) b k g k (u k l ) Weight of flow s of class k Throughput sensi*vity of class k, e.g., log(.) Total sending rate of flow s of class k Delay sensi*vity of class k U*liza*on of class k over link l Sum of the products of path rates and average end- to- end delays on those paths

9 Model and Formula*on Objec*ve Func*on q Data centers, backbone and traffic sources under the same OSP ownership q Maximize the sum of u*li*es of all flows across all traffic classes (global social welfare ) Global Problem G: maximize U = X k X s2f k U k s subject to AR k z k y k, 8k X yl k apple c l, 8l variables z k 0, 8k k

10 Two- Tier Design Each controller has a limited view about the network and inter- DC traffic Network y k Link Coordinator (LC) Op*mizes aggregate link bandwidth across classes A centralized en*ty Knows network topology Coordina*on C1... Classes CK z k Class Allocator (CA) Op*mizes sending rates across flows in its own class One for each class Knows the u*lity func*on, weights, and paths of individual flows in its own class F F F F F F F Flows

11 Two- Tier Design Network Coordina*on C1... Classes CK Tier-1 Link Coordinator 1 2 Tier-2 Class Allocator CA 1 Message- passing... CA k... CA K 3 Class Allocator 3 3 Class Allocator Message- passing DC 1 Sources DC j DC J F F F F F F F Flows

12 Two- Tier Decomposi*on Primal Decomposi*on Tier-1 MASTER PRIMAL Link Coordinator Coordinates all the subproblems Tier-2 SUBPROBLEM CLASS(1)... SUBPROBLEM... CLASS(k) SUBPROBLEM CLASS(K) Class Allocator Solves independently

13 Two- Tier Decomposi*on Primal Decomposi*on Tier-1 MASTER PRIMAL Link Coordinator Coordinates all the subproblems Tier-2 SUBPROBLEM CLASS(1)... SUBPROBLEM... CLASS(k) SUBPROBLEM CLASS(K) Class Allocator Solves independently Subproblem for Class k maximize U k = X s2f k U k s subject to AR k z k y k 8k variables z k 0

14 Two- Tier Decomposi*on Primal Decomposi*on Tier-1 MASTER PRIMAL Link Coordinator Coordinates all the subproblems Tier-2 SUBPROBLEM CLASS(1)... SUBPROBLEM... CLASS(k) SUBPROBLEM CLASS(K) Class Allocator Solves independently Subproblem for Class k maximize U k = X s2f k U k s subject to AR k z k y k 8k variables z k 0 Master Primal maximize U = X U k (y k ) k X subject to yl k apple c l 8l k variables y k 0

15 Two- Tier Decomposi*on Primal Decomposi*on Tier-1 MASTER PRIMAL Link Coordinator Coordinates all the subproblems Tier-2 SUBPROBLEM CLASS(1)... SUBPROBLEM... CLASS(k) SUBPROBLEM CLASS(K) Class Allocator Solves independently Message- Passing Link Coordinator l k* y k step size : Class Allocator CAk zk sp y k : Aggregate bandwidth assigned to class k k : Optimal subgradient of CLASS(k)

16 Three- Tier Design Why another *er? (High control overhead) q Flow of a given class may originate from any DC Network q Each class allocator poten*ally communicates with all DCs C1... Classes CK F F F F F F F Flows

17 Three- Tier Design Why another *er? (High control overhead) q Flow of a given class may originate from any DC Network q Each class allocator poten*ally communicates with all DCs C1... Classes CK C1... Classes CK DC1... DCj... DCJ Data centers F F F F F F F Flows F F F F F F F Flows

18 Three- Tier Design One centralized en*ty Link Coordinator (LC) Op*mizes aggregate link bandwidth across classes y k Network One per class Class Allocator (CA) Op*mizes aggregate link bandwidth across DCs sending traffic in its own class y kj C1... Classes CK Data Center Allocator (DCA) Op*mizes sending rates across flows in its own class origina*ng from its own DC One per class, per DC z k DC1... DCj... DCJ F F F F F F F Flows Data centers

19 Three- Tier Design Tier-1 Link Coordinator Tier-2 Class Allocator CA Class Allocator Message- passing Class Allocator... CA k... CA K Tier-3 DC Allocator DA k DC Allocator DA kj DC Allocator DA kj Message- passing DC 1 Sources 5 DC j Message- passing DC J

20 Three- Tier Decomposi*on 2- Level Primal Decomposi*on Tier-2 SECONDARY-PRIMAL CLASS(1) Tier-3 Tier-1 SUBPROBLEM DATACENTER(k,1) MASTER-PRIMAL SECONDARY-PRIMAL CLASS(k)... SUBPROBLEM... DATACENTER(k,j) Link Coordinator Coordinates the secondary primals SECONDARY-PRIMAL CLASS(K) SUBPROBLEM DATACENTER(K,J) Class Allocator Coordinates the subproblems of its own class Data Center Allocator Solves independently

21 Three- Tier Decomposi*on Message- Passing kj s k : Optimal subgradient of CLASS(k) : Optimal subgradient of DATACENTER(k,j) Link Coordinator sk* Outer loop slower y k Class Allocator CAk l kj* Inner loop faster y kj Data Center Allocator DAkj zk sp y k : Aggregate bandwidth assigned to class k y kj : Aggregate bandwidth assigned to DC j sending tra c of class k

22 Performance Evalua*on Performance Metrics q Rate of convergence q Message- passing overhead DC2 1 l1 l3 3 DC3 l2 DC1 2 Simple topology DC1 & DC2 send traffic to DC3 100 Mbps link capacity Two classes with log u*lity DC3 DC4 10 DC2 Abilene topology 4 DCs 1 Gbps link capacity in each direc*on First 3 shortest possible paths between every pair of DCs (36 total) Two classes with log u*lity func*ons DC1 1 2

23 Rate of Convergence % error Class-level step size : =1 =2 =5 =10 =20 % error Class-level step size : DC-level step size : =2 =1 =2 =5 =10 = # of iterations to convergence Two- *er # of iterations to convergence Three- *er Each itera*on is in the order of a few seconds: q There can be only so many different types of performance requirements (~10) q Only so many inter- connected DCs (a few 10s)

24 Rate of Convergence # of iterations to convergence =2 =4 =6 = Three- *er: Number of itera*ons to converge for different combina*ons of class- level and DC- level step sizes.

25 Rate of Convergence Summary of the convergence behavior q In prac*ce, choose step sizes that converge quickly. q Dynamic traffic demand: For private OSP backbone, the demand variability can be controlled to some extent

26 Message- Passing Overhead (Un*l convergence) # of messages exchanged 15 x tier design 3 tier design # of flows in each class per data center q Messages are sent over the wide area network q Number of messages depends on the number of flows in the two- *er design, but not in the three- *er design q Small compared to the total traffic volume 0 1 Two- *er: + X X X X R k A Three- *er: N 0 (2KL +2JKLM) sp # of variables # of variables k j p s2f kj K L J N N M 5 No. of classes No. of backbone links No. of DCs No. of class- level alloca*ons to converge in two- *er and three- *er designs No. of DC- level alloca*ons to converge in three- *er

27 Conclusions q SoRware defined traffic management for wide area data center backbone networks q Two scalable and prac*cal semi- centralized designs using a small number of controllers that can be implemented in real- world data center backbones (Google) q Joint rate control, rou*ng, and link scheduling using op*miza*on in a modular, *ered design q Results provably op*mal using principles of op*miza*on decomposi*on q Tradeoff between rate of convergence and message- passing choose the design that suits the OSP best

28 Thank You q Amitabha Ghosh, Sangtae Ha, Edward Crabbe, and Jennifer Rexford, Scalable Mul*- Class Traffic Management in Data Center Backbone Networks, IEEE JSAC: Networking Challenges in Cloud Compu*ng Systems and Applica*ons, vol. 13, no. 12, 2013 (in press). hsp://anrg.usc.edu/~amitabhg/papers/jsac- CloudCompu*ng pdf