Beyond the Stars: Revisiting Virtual Cluster Embeddings

Transcription

1 Beyond the Stars: Revisiting Virtual Cluster Embeddings Matthias Rost Technische Universität Berlin September 7th, 2015, Télécom-ParisTech Joint work with Carlo Fuerst, Stefan Schmid Published in ACM SIGCOMM CCR Volume 45 Issue 3, July 2015 Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

2 Introduction 1 Introduction Problem Space: Running Applications in the Cloud Application Abstractions: The VC Abstraction 2 The classical VC Embedding Problem Overview Definition: VC Embedding Problem VC-ACE Algorithm 3 Hose-Based Virtual Cluster Embeddings Motivation Definition: Hose-Based VC Embedding Problem Computational Complexity of HVC Embeddings Computing (Fractional) HVC Embeddings 4 Computational Evaluation Experimental Setup Results Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

3 Introduction Problem Space: Running Applications in the Cloud Data Center Applications Virtualization has changed our view on complex computations Virtual machines (VMs) can be spawned within minutes and anywhere on the world (Microsoft Azure, Amazon EC2,... ) Complex tasks can quite easily be distributed to tens or hundreds of servers using frameworks like MapReduce Insight: Application performance does depend on bandwidth availability Facebook traces show that network transfers account for 33% of the execution time [4] Data centers exhibit oversubscription factors of upto 1:240 [6] Customer s application execution time can hardly be estimated! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

4 Introduction Problem Space: Running Applications in the Cloud Data Center Applications Insight Performance does depend on bandwidth availability How to achieve resource isolation? New topologies limiting the oversubscription factor: Fat trees, MDCubes,... Figure: Fat tree topology [1] Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

5 Introduction Problem Space: Running Applications in the Cloud Data Center Applications Insight Performance does depend on bandwidth availability How to achieve resource isolation? New topologies limiting the oversubscription factor: Fat trees, MDCubes,... Figure: Fat tree topology [1] Novel descriptions of applications embedding algorithms Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

6 Introduction Problem Space: Running Applications in the Cloud Application Specification : Case Study Amazon AWS Customers can select from a huge number of instances Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

7 Introduction Problem Space: Running Applications in the Cloud Application Specification : Case Study Amazon AWS Customers can select from a huge number of instances Customers can select enhanced networking or cluster networking : Instances launched into a common cluster placement group are placed into a logical cluster that provides high-bandwidth, low-latency networking between all instances in the cluster. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

8 Introduction Problem Space: Running Applications in the Cloud Application Specification : Case Study Amazon AWS Customers can select from a huge number of instances Customers can select enhanced networking or cluster networking : Instances launched into a common cluster placement group are placed into a logical cluster that provides high-bandwidth, low-latency networking between all instances in the cluster. Resource Isolation? Best-effort! Level of Specification? Grouping of instances! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

9 Application Abstractions: The VC Abstraction

10 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction Early 2000s: Virtual Network Embedding Problem Requests are specified as graphs Nodes represent VMs Edges represent inter-vm links - CPU - RAM - Disk - latency - bandwidth Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

11 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction Early 2000s: Virtual Network Embedding Problem Requests are specified as graphs Nodes represent VMs Edges represent inter-vm links - CPU - RAM - Disk - latency - bandwidth Pro Concise specification Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

12 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction Early 2000s: Virtual Network Embedding Problem Requests are specified as graphs Nodes represent VMs Edges represent inter-vm links - CPU - RAM - Disk - latency - bandwidth Pro Concise specification Contra Do customers know their requirements? Generally: Challenging NP-hard problem! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

13 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction 2011: Virtual Cluster (VC) Abstraction [3] Allows only for star -shaped graphs VMs are connected to logical switch logical switch VMs Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

14 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction 2011: Virtual Cluster (VC) Abstraction [3] Allows only for star -shaped graphs VMs are connected to logical switch logical switch VMs Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

15 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction 2011: Virtual Cluster (VC) Abstraction [3] Allows only for star -shaped graphs VMs are connected to logical switch Requests are specified by three parameters: N N number of virtual machines C N size of virtual machines B R + bandwidth towards logical switch C 1 B B 2 B N C C... C Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

16 Introduction Application Abstractions: The VC Abstraction The Right Level of Abstraction 2011: Virtual Cluster (VC) Abstraction [3] Allows only for star -shaped graphs VMs are connected to logical switch Requests are specified by three parameters: N N number of virtual machines C N size of virtual machines B R + bandwidth towards logical switch C 1 B B 2 B N C C... C Pro Simple specification! Well-performing heuristics for data-center topologies [3, 8] Contra The VM size and the amount of bandwidth are dictated by the maximum wasteful Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

17 Introduction Application Abstractions: The VC Abstraction On Traffic Matrices Graph Abstraction VC Abstraction B {1,2} 1 B {1,3} 2 3 B {2,3} 1 B B 2 B 3 Allows for any traffic matrix M, where the bandwidth for edge {i, j} is less than B {i,j}. Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

18 The classical VC Embedding Problem Overview Outlook Previous works... only considered (fat) trees only considered heuristics Ballani et al.: Oktopus [3] allocating virtual cluster requests on graphs with bandwidthconstrained edges is NP-hard Xie et al.: Proteus [8] [Our algorithm] picks the first fitting lowest-level subtree out of all such lowest-level subtrees. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

19 The classical VC Embedding Problem Overview Outlook Previous works... only considered (fat) trees only considered heuristics Ballani et al.: Oktopus [3] allocating virtual cluster requests on graphs with bandwidthconstrained edges is NP-hard Xie et al.: Proteus [8] [Our algorithm] picks the first fitting lowest-level subtree out of all such lowest-level subtrees. Main Questions Is the VC embedding problem really NP-hard to solve? Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

20 Formal Definition of the VC Embedding Problem

21 The classical VC Embedding Problem Definition: VC Embedding Problem VC Embedding Problem Defintion VC request: N, B, C VC = (V VC, E VC ), V VC = {1, 2,..., N, center} E VC = {{i, center} 1 i N } Physical Network (Substrate) S = (V S, E S, cap, cost), cap : V S E S N cost : V S E S R 0 Task: Find a mapping of... VMs onto substrate nodes map V : V VC V S, and VC edges onto paths in the substrate map E : E VC P(E S ) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

22 The classical VC Embedding Problem Definition: VC Embedding Problem VC Embedding Problem Defintion VC request: N, B, C VC = (V VC, E VC ), V VC = {1, 2,..., N, center} E VC = {{i, center} 1 i N } Physical Network (Substrate) S = (V S, E S, cap, cost), cap : V S E S N cost : V S E S R 0 Task: Find a mapping of... VMs onto substrate nodes map V : V VC V S, and VC edges onto paths in the substrate map E : E VC P(E S ), such that 1 map E ({u, v}) connects map V (u) and map V (v) for {u, v} E VC Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

23 The classical VC Embedding Problem Definition: VC Embedding Problem VC Embedding Problem Defintion VC request: N, B, C VC = (V VC, E VC ), V VC = {1, 2,..., N, center} E VC = {{i, center} 1 i N } Physical Network (Substrate) S = (V S, E S, cap, cost), cap : V S E S N cost : V S E S R 0 Task: Find a mapping of... VMs onto substrate nodes map V : V VC V S, and VC edges onto paths in the substrate map E : E VC P(E S ), such that 1 map E ({i, j}) connects map V (i) and map V (j) for {i.j} E VC 2 v V VC \ {center} v = map V (v ) C cap(v) and e E VC e map E (e ) B cap(e) for v V S, e E S Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

24 The classical VC Embedding Problem Definition: VC Embedding Problem VC Embedding Problem Defintion Task: Find a mapping of... VMs onto substrate nodes map V : V VC V S, and VC edges onto paths in the substrate map E : E VC P(E S ), such that 1 map E ({u, v}) connects map V (u) and map V (v) for {u, v} E VC 2 v V VC \ {center} v = map V (v ) C cap(v) and 3 minimizing the cost C e E VC e map E (e ) v V VC \{center} B cap(e) for v V S, e E S cost(map V (v)) + B e E VC e map E (e ) cost(e). Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

25 VC-ACE Algorithm

26 The classical VC Embedding Problem VC-ACE Algorithm Key Insights Lemma We can assume B = C = 1. Proof idea. If B 1, C 1, we transform the substrate by scaling capacities and costs: cap S (u) = cap(u)/c for u V S cap S (e) = cap(e)/b for e E S cost S (u) = cost(u) C for u V S cost S (e) = cost(e) B for e E S Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

27 The classical VC Embedding Problem VC-ACE Algorithm Key Insights Lemma We can solve the edge embedding problem if all nodes are placed. Proof. 1 Construct extended graph with additional node s + and (parallel) edges: {(s +, map V (i)) i {1,..., N }} of capacity 1 and cost 0 2 Compute a minimum cost flow of value N from s + to map V (center). 3 Perform a path-decomposition to obtain mapping for edges. 2 a b a b 1 source s + 3 d c d target c Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

28 The classical VC Embedding Problem VC-ACE Algorithm Key Insights Lemma We can solve the embedding problem if the logical switch is placed. Proof. 1 Construct extended graph with additional edges {(s +, u) u V S }, cap(s +, u) = cap(u) and cost(s +, u) = cost(u) for u V S 2 Compute a minimum cost flow of value N from s + to map V (center). 3 Perform path-decomposition to obtain mapping for nodes and edges. 2? cap(a) = 3 cap(b) = 2 a b a b 1? source s + 3? d c cap(d) = 1 cap(c) = 1 d target c Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

29 The classical VC Embedding Problem VC-ACE Algorithm Key Insights Lemma We can solve the embedding problem if the logical switch is placed. Proof. 1 Construct extended graph with additional edges {(s +, u) u V S }, cap(s +, u) = cap(u) and cost(s +, u) = cost(u) for u V S 2 Compute a minimum cost flow of value N from s + to map V (center). 3 Perform path-decomposition to obtain mapping for nodes and edges. flow = 2 a b 2 a b source s + flow = 2 1 flow = 1 d target c 3 d c Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

30 The classical VC Embedding Problem VC-ACE Algorithm VC-ACE Algorithm Idea Simply iterate over possible locations for the center. Algorithm 1: The VC-ACE Algorithm Input: Substrate S = (V S, E S ), request (N, B, C) Output: Optimal VC mapping map V, map E if feasible (ˆf, ˆv) (null, null) for v V S do V S = V S {s + } and E S = E S {(s +, u) u V S } cap { S (e) = cap(e)/b, if e E S cap(u)/c, if e = (s +, u) E { S cost(e) B, if e E S cost S (e) = cost(u) C, if e = (s +, u) E S f MinCostFlow(s +, v, N, V S, E S, cap S, cost S ) if f is feasible and cost(f ) < cost(ˆf ) then (ˆf, ˆv) (f, v) if ˆf = null then return null return DecomposeFlowIntoMapping(ˆf, ˆv) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

31 The classical VC Embedding Problem VC-ACE Algorithm VC-ACE Algorithm Idea Simply iterate over possible locations for the center. Theorem Correctness follows from the lemma on the previous slide. Algorithm 2: The VC-ACE Algorithm Input: Substrate S = (V S, E S ), request (N, B, C) Output: Optimal VC mapping map V, map E if feasible (ˆf, ˆv) (null, null) for v V S do V S = V S {s + } and E S = E S {(s +, u) u V S } cap { S (e) = cap(e)/b, if e E S cap(u)/c, if e = (s +, u) E { S cost(e) B, if e E S cost S (e) = cost(u) C, if e = (s +, u) E S f MinCostFlow(s +, v, N, V S, E S, cap S, cost S ) if f is feasible and cost(f ) < cost(ˆf ) then (ˆf, ˆv) (f, v) if ˆf = null then return null return DecomposeFlowIntoMapping(ˆf, ˆv) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

32 The classical VC Embedding Problem VC-ACE Algorithm VC-ACE Algorithm Theorem The runtime is O ( N (n 2 log n + n m) ) with n = V S and m = E S, when using the successive-shortest path for the flow computation. Corollary. The VC Embedding Problem is not NP-hard. Algorithm 3: The VC-ACE Algorithm Input: Substrate S = (V S, E S ), request (N, B, C) Output: Optimal VC mapping map V, map E if feasible (ˆf, ˆv) (null, null) for v V S do V S = V S {s + } and E S = E S {(s +, u) u V S } cap { S (e) = cap(e)/b, if e E S cap(u)/c, if e = (s +, u) E { S cost(e) B, if e E S cost S (e) = cost(u) C, if e = (s +, u) E S f MinCostFlow(s +, v, N, V S, E S, cap S, cost S ) if f is feasible and cost(f ) < cost(ˆf ) then (ˆf, ˆv) (f, v) if ˆf = null then return null return DecomposeFlowIntoMapping(ˆf, ˆv) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

33 We can compute optimal solutions in polynomial-time.

34 We can compute optimal solutions in polynomial-time. Can we do even better?

35 Can we do even better? Yes,...

36 Hose-Based Virtual Cluster Embeddings

37 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction 1 B B 2 B 3 Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

38 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction 1 B B 2 B 3 Question: What is the purpose of the switch? Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

39 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction 1 B B 2 B Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B 3 Question: What is the purpose of the switch? Ballani et al. Oktopus [3] Oktopus allocation algorithms assume that the traffic between a tenant s VMs is routed along a tree. Answer: To route the traffic along a tree. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

40 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction 1 B B 2 B 3 Question: Can we do without the switch? Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

41 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction Question: Can we do without the switch? 1 B B 2 B 3 Ballani et al. Oktopus [3] Alternatively, the NM [Network Manager] can control datacenter routing to actively build routes between tenant VMs [..] Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B We defer a detailed study of the relative merits of these approaches to future work. Answer: Yes! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

42 Hose-Based Virtual Cluster Embeddings Motivation Starting from Scratch VC Abstraction Hose-Based VC Abstraction 1 B 2 B B 1 2 B B 3 B 3 Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Allows for any traffic matrix M, where for any VM the sum of outgoing and incoming traffic is less than B Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

43 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example cap = 1 cap = N = 6, B = C = 1 ring substrate Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

44 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example cap = 1 cap = N = 6, B = C = 1 ring substrate There exists no solution in the classic VC embedding model. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

45 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example allocation = N = 6, B = C = 1 6 Embedding on the same substrate 5 Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

46 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example allocation = N = 6, B = C = 1 Embedding on the same substrate There exists a solution in the hose-based VC model! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

47 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example allocation = 2 5 Embedding on the same substrate 4 Why allocations of 2 are sufficient: Consider edge e between VMs 6 and 5. The edge is used by routes R(e) = {(1, 5), (2, 5), (3, 6), (4, 6), (5, 6)}. Any valid traffic matrix M will respect: M 1,5 + M 2,5 1 M 3,6 + M 4,6 + M 5,6 1 Hence (i,j) R(e) M i,j 2 holds. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

48 Hose-Based Virtual Cluster Embeddings Motivation Motivating Example II cap = 0 cap = 1 cap = 2 cap = cost = 6 5 N = 6, B = C = 1 extended ring topology Solution costs can be arbitrarily higher under the classic star-interpretation! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

49 Hose-Based Virtual Cluster Embedding Problem

50 Hose-Based Virtual Cluster Embeddings Definition: Hose-Based VC Embedding Problem Hose-Based VC Embedding Problem (HVCEP) Definition (Clique Graph) V C = {1,..., N }, E C = {(i, j) i, j V C, i < j} Task: Find a mapping of... VC nodes onto substrate nodes map V : V C V S, and VC routes onto paths in the substrate map E : E C P(E S ), such that 1 route (i, j) E C connects map V (i) and map V (j), 2 the mapping of VMs must not violate node capacities (cf. slide 12), Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

51 Hose-Based Virtual Cluster Embeddings Definition: Hose-Based VC Embedding Problem Hose-Based VC Embedding Problem (HVCEP) Definition (Clique Graph) V C = {1,..., N }, E C = {(i, j) i, j V C, i < j} Task: Find a mapping of... VC nodes onto substrate nodes map V : V C V S, and VC routes onto paths in the substrate map E : E C P(E S ), and integral bandwidth reservations l u,v cap(u, v) for {u, v} E S, s.t. 1 route (i, j) E C connects map V (i) and map V (j), 2 the mapping of VMs must not violate node capacities (cf. slide 12), 3 for all valid traffic matrices M ij i.e. (j,i) E C M ji + M ij B holds the bandwidth reservation is not exceeded on any edge {u, v} E S : {i,j} E C :{u,v} map E ({i,j}) M ij l u,v, Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

52 Hose-Based Virtual Cluster Embeddings Definition: Hose-Based VC Embedding Problem Hose-Based VC Embedding Problem (HVCEP) Definition (Clique Graph) V C = {1,..., N }, E C = {(i, j) i, j V C, i < j} Task: Find a mapping of... VC nodes onto substrate nodes map V : V C V S, and VC routes onto paths in the substrate map E : E C P(E S ), and integral bandwidth reservations l u,v cap(u, v) for {u, v} E S, such that 1 route (i, j) E C connects map V (i) and map V (j), 2 the mapping of VMs must not violate node capacities (cf. slide 12), 3 for all valid traffic matrices M i.e. (j,i) E C M ji + M ij B holds the bandwidth reservation is not exceeded on any edge {u, v} E S : {i,j} E C :{u,v} map E ({i,j}) M ij l u,v, 4 minimizing C cost(map V (i)) + B l u,v cost(e). i V C e E S Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

53 Computational Complexity of HVC Embeddings

54 Hose-Based Virtual Cluster Embeddings Computational Complexity of HVC Embeddings Excursion: VPN Embeddings and the VPN Conjecture Definition (VPN Embedding Problem (VPNEP) [7]) Given: Substrate network G = (V, E) with edge costs cost : E R + 0 Set of terminals W V with demands b(i) R + for i W Task: Find paths P {i,j} for all pairs i, j W, i j, and bandwidth allocations x e R + 0 on edges e E, s.t. i,j W :i j,p {i,j} M {i,j} x e holds for traffic matrices M, minimizing the cost e E cost(e) x e. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

55 Hose-Based Virtual Cluster Embeddings Computational Complexity of HVC Embeddings Excursion: VPN Embeddings and the VPN Conjecture Definition (VPN Embedding Problem (VPNEP) [7]) Given: Substrate network G = (V, E) with edge costs cost : E R + 0 Set of terminals W V with demands b(i) R + for i W Task: Find paths P {i,j} for all pairs i, j W, i j, and Theorem bandwidth allocations x e R + 0 on edges e E, s.t. i,j W :i j,p {i,j} M {i,j} x e holds for traffic matrices M, minimizing the cost e E cost(e) x e. Finding a feasible solution for the capacitated VPNEP is NP-hard [7]. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

56 Hose-Based Virtual Cluster Embeddings Computational Complexity of HVC Embeddings Excursion: VPN Embeddings and the VPN Conjecture Definition (VPN Embedding Problem (VPNEP) [7]) Given: Substrate network G = (V, E) with edge costs cost : E R + 0 Set of terminals W V with demands b(i) R + for i W Task: Find paths P {i,j} for all pairs i, j W, i j, and Theorem bandwidth allocations x e R + 0 on edges e E, s.t. i,j W :i j,p {i,j} M {i,j} x e holds for traffic matrices M, minimizing the cost e E cost(e) x e. Finding a feasible solution for the capacitated VPNEP is NP-hard [7]. Theorem (By reduction from the VPNEP) Finding a feasible solution for the HVCEP is NP-hard. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

57 Hose-Based Virtual Cluster Embeddings Computational Complexity of HVC Embeddings Excursion: VPN Embeddings and the VPN Conjecture Definition (VPN Embedding Problem (VPNEP) [7]) Given: Substrate network G = (V, E) with edge costs cost : E R + 0 Set of terminals W V with demands b(i) R + for i W Task: Find paths P {i,j} for all pairs i, j W, i j, and bandwidth allocations x e R + 0 on edges e E, s.t. i,j W :i j,p {i,j} M {i,j} x e holds for traffic matrices M, minimizing the cost e E cost(e) x e. Theorem (VPN Conjecture) Tree routing and arbitrary routing solutions coincide for the VPNEP on uncapacitated graphs. [5]. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

58 Hose-Based Virtual Cluster Embeddings Computational Complexity of HVC Embeddings Excursion: VPN Embeddings and the VPN Conjecture Definition (VPN Embedding Problem (VPNEP) [7]) Given: Substrate network G = (V, E) with edge costs cost : E R + 0 Set of terminals W V with demands b(i) R + for i W Task: Find paths P {i,j} for all pairs i, j W, i j, and bandwidth allocations x e R + 0 on edges e E, s.t. i,j W :i j,p {i,j} M {i,j} x e holds for traffic matrices M, minimizing the cost e E cost(e) x e. Theorem (VPN Conjecture) Tree routing and arbitrary routing solutions coincide for the VPNEP on uncapacitated graphs. [5]. Theorem (Via the VPN Conjecture) Algorithm VC-ACE solves the HVCEP when capacities are sufficiently large! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

59 Computing (Fractional) HVC Embeddings

60 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings A Mixed-Integer Programming Formulation for the HVCEP Variables x i u y i,j u.v l u.v mapping of VM i onto node u mapping of link (i, j) V C onto (directed) substrate edge (u, v) load on substrate edge {u, v} ω i uv dual variable for allocation of communications of VM i on edge {u, v} Mixed-Integer Program 1: HVC-OSPE min cost u xu i + cost u,v l uv (1) i V C,u V S {u,v} E S xu= i 1 i V C. (2) u V S σ u (xu i xu i+1 ) 0 i V C \ {N }. (3) u V S C xu i cap u u V S. (4) i V C (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (5) y ij vu = x i u x j u (i, j) E C, (6) u V S. i V C B ω i uv l uv {u, v} E S. (7) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (8) {u, v} E S. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

61 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings A Mixed-Integer Programming Formulation for the HVCEP Explanation (2) - (4) control the VM embedding (5) - (8) is adapted from Altin et al. [2] for computing the optimal hose allocations on edges Mixed-Integer Program 2: HVC-OSPE min cost u xu i + cost u,v l uv (1) i V C,u V S {u,v} E S xu= i 1 i V C. (2) u V S σ u (xu i xu i+1 ) 0 i V C \ {N }. (3) u V S C xu i cap u u V S. (4) i V C (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (5) y ij vu = x i u x j u (i, j) E C, (6) u V S. i V C B ω i uv l uv {u, v} E S. (7) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (8) {u, v} E S. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

62 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings A Mixed-Integer Programming Formulation for the HVCEP Explanation (2) - (4) control the VM embedding (5) - (8) is adapted from Altin et al. [2] for computing the optimal hose allocations on edges Observation There are O(N 2 E S ) binary variables for computing paths. Mixed-Integer Program 3: HVC-OSPE min cost u xu i + cost u,v l uv (1) i V C,u V S {u,v} E S xu= i 1 i V C. (2) u V S σ u (xu i xu i+1 ) 0 i V C \ {N }. (3) u V S C xu i cap u u V S. (4) i V C (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (5) y ij vu = x i u x j u (i, j) E C, (6) u V S. i V C B ω i uv l uv {u, v} E S. (7) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (8) {u, v} E S. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

63 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings A Mixed-Integer Programming Formulation for the HVCEP Observation There are O(N 2 E S ) binary variables for computing paths. Initial Computational Results Solving this formulation may take up to 1800 seconds for embedding a 10-VM VC onto a 20 node substrate. Mixed-Integer Program 4: HVC-OSPE min cost u xu i + cost u,v l uv (1) i V C,u V S {u,v} E S xu= i 1 i V C. (2) u V S σ u (xu i xu i+1 ) 0 i V C \ {N }. (3) u V S C xu i cap u u V S. (4) i V C (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (5) y ij vu = x i u x j u (i, j) E C, (6) u V S. i V C B ω i uv l uv {u, v} E S. (7) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (8) {u, v} E S. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

64 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings A Mixed-Integer Programming Formulation for the HVCEP Further Observations The hardness result has shown that the problem is hard even if the VMs are fixed. The large number of variables necessary for computing each end-to-end path between VMs renders solving even the linear relaxation i.e. dropping integrality constraints computationally hard. Assumptions for obtaining a solvable formulation Assume that the VMs are already mapped. Assume that the hose-paths are splittable, i.e. each VMs are connected by a set of (weighted) paths. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

65 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings Assumptions Assume that the VMs are already mapped. Assume that the hose-paths are splittable. arbitrarily many paths. map V (i) map V (j) y ij e = 0.5 Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

66 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (5) y ij vu = x i u x j u (i, j) E C, (6) u V S. i V C B ω i uv l uv {u, v} E S. (7) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (8) {u, v} E S. map V (i) map V (j) W e δ + (W ) y ij e = 1 This type of constraint is equivalent to (6). Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

67 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings l uv cap uv {u, v} E S. (5) yuv ij yvu ij = x u i x j (i, j) E u C, (6) (u,v) δ u + (v,u) δu u V S. B ωuv i l uv {u, v} E S. (7) i VC y ij uv + y ij vu ωi uv + ω j uv (u,v) δ + (W ) y ij uv 1 Derivation of a new constraint (i, j) E C, (8) {u, v} E S. (i, j) E C. W V S : map V (i) W, (6 ) map V (j) / W Across a cut W, the amount of flow must be greater than 1 (6 ). By summing up Constraints (8) accordingly, we obtain for (i, j) E C and across any node set W V S with map V (i) W and map V (j) / W that (ωuv i + ωuv) j 1 holds. (u,v) δ + (W ) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

68 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings Remarks l uv cap uv {u, v} E S. (5) yuv ij yvu ij = x u i x j (i, j) E u C, (6) (u,v) δ u + (v,u) δu u V S. B ωuv i l uv {u, v} E S. (7) (u,v) δ + (W ) i VC y ij uv + y ij vu ωi uv + ω j uv (u,v) δ + (W ) y ij uv 1 (ω i uv + ω j uv) 1 (i, j) E C, (8) {u, v} E S. (i, j) E C. W V S : map V (i) W, (6 ) map V (j) / W (i, j) E C. W V S : map V (i) W, (9) map V (j) / W Given (9), we can always (re-)construct the flow variables y ij uv afterwards by breadth-first searches. Furthermore, this property does not depend on (6 ). Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

69 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings l uv cap uv {u, v} E S. (5) yuv ij yvu ij = x u i x j (i, j) E u C, (6) (u,v) δ u + (v,u) δu u V S. B ωuv i l uv {u, v} E S. (7) (u,v) δ + (W ) i VC y ij uv + y ij vu ωi uv + ω j uv (u,v) δ + (W ) y ij uv 1 (ω i uv + ω j uv) 1 (i, j) E C, (8) {u, v} E S. (i, j) E C. W V S : map V (i) W, (6 ) map V (j) / W (i, j) E C. W V S : map V (i) W, (9) map V (j) / W Remarks Therfore Constraints (6), (8), and (6 ) are not needed anymore! Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

70 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings (u,v) δ + u y ij uv (v,u) δ u l uv cap uv {u, v} E S. (6) y ij vu = x i u x j u (i, j) E C, (7) u V S. i V C B ω i uv l uv {u, v} E S. (8) y ij uv + y ij vu ωi uv + ω j uv (i, j) E C, (9) {u, v} E S. Algorithm 5: HMPR min cost u,v l uv (10) {u,v} E S l uv cap uv {u, v} E S. (11) B ωuv i l uv {u, v} E S. (12) i V C (ωuv i + ωuv) j (i, j) E C. W V S : 1 map V (i) W, (13) map V (j) / W (u,v) δ + (W ) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

71 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings Algorithm 6: HMPR min cost u,v l uv (10) {u,v} E S l uv cap uv {u, v} E S. (11) B ωuv i l uv {u, v} E S. (12) i V C (ωuv i + ωuv) j (i, j) E C. W V S : 1 map V (i) W, (13) map V (j) / W (u,v) δ + (W ) Exponential number of constraints, which can be separated in polynomial time. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

72 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings Algorithm 7: HMPR min cost u,v l uv (10) {u,v} E S l uv cap uv {u, v} E S. (11) B ωuv i l uv {u, v} E S. (12) i V C (ωuv i + ωuv) j (i, j) E C. W V S : 1 map V (i) W, (13) map V (j) / W (u,v) δ + (W ) Exponential number of constraints, which can be separated in polynomial time. Number of variables, in the order of O(N E S ) instead of O(N 2 E S ). Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

73 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings We can compute fractional edge embeddings, but how to find node locations? Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

74 Hose-Based Virtual Cluster Embeddings Computing (Fractional) HVC Embeddings Computing Splittable HVC Embeddings Heuristic Idea without capacities: VC = HVC reuse VC-ACE algorithm, but allow violation of capacities w.r.t. VC model violating capacities induces k times the cost of the original edge Algorithm 5: The HVC-ACE Embedding Heuristic Input: Substrate S = (V S, E S ), request VC(N, B, C), cost factor k 1 Output: Splittable HVC-Embedding map V, map E E S for e E S do E S = E S {e, e } cap S (e) = cap(e) and cap S (e ) = cost S (e) = cost(e) and cost S (e ) = cost(e) k map V, map E VC-ACE(V S, E S, VC(N, B, C)) if map V null then map E HMPR(VC(N, B, C), map V ) if map E null then return map V, map E return null Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

75 Computational Evaluation What do we get by using HVC-ACE?

76 Computational Evaluation Experimental Setup Setup Topologies Fat trees with 12 port switches and 432 server overall MDCubes consisting of 4 BCubes with 12 port switches and k = 1, such that the topology contains 576 server Figure: Fat tree (n=4) Figure: MDCube (n=2,k=1) Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

77 Computational Evaluation Experimental Setup Setup Generation of Requests N is chosen uniformly at random from the interval {10,..., 30}. B is uniformly distributed in the interval of {20%,..., 100%} w.r.t. to the available capacity of an unused link. C = 1 and the capacity of servers are 2. Generation of Scenarios Requests are embedded over time using the VC-ACE algorithm. After system stabilization, a single data point is generated by considering the performance of both algorithms on the same substrate state and the same request. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

78 Computational Evaluation Experimental Setup Metrics Acceptance Ratio How many requests can VC-ACE embed compared to HVC-ACE? Footprint Change Assuming that both algorithms have found a solution, how much resources do we save by using HVC-ACE (compared to VC-ACE using 100%). Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

79 Results

80 Computational Evaluation Results Results on Fat Tree Topology Acceptance Ratio [%] Request Size VC ACE HVC ACE HVC-ACE can improve acceptance ratio dramatically. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

81 Computational Evaluation Results Results on Fat Tree Topology ECDF(Requests) Footprint Change [%] HVC-ACE saves > 10% of resources for more than 20% of the requeusts. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

82 Computational Evaluation Results Results on MDCube Acceptance Ratio [%] Request Size VC ACE HVC ACE HVC-ACE can improve acceptance by around 20% on average. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

83 Computational Evaluation Results Results on MDCube ECDF(Requests) Footprint Change [%] HVC-ACE saves no resources. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

84 Computational Evaluation Results Conclusion Contributions Showed how to solve the classic VC embedding problem optimally. Defined formally the hose-based VC embedding problem and considered its computational complexity. Derived a compact formulation for the splittable hose edge embedding and the HVC-ACE heuristic for the respective embedding problem. Showed that by using HVC-ACE one may indeed save a substantial amount of resources and increase the acceptance ratio (on fat trees). Bottomline Complexity of specification can often be traded-off with the complexity of the respective embedding algorithms. We need to understand this trade-off better and explore the boundaries of specifications that we can efficiently embed. Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

85 Computational Evaluation Results References I [1] M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In ACM SIGCOMM Computer Communication Review, volume 38, pages ACM, [2] A. Altin, E. Amaldi, P. Belotti, and M. c. Pinar. Provisioning virtual private networks under traffic uncertainty. Networks, 49(1): , Jan [3] H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron. Towards predictable datacenter networks. In Proc. ACM SIGCOMM, [4] M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica. Managing Data Transfers in Computer Clusters with Orchestra. In ACM SIGCOMM, [5] N. Goyal, N. Olver, and F. B. Shepherd. The VPN conjecture is true. Proc. ACM STOC, Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September

86 Computational Evaluation Results References II [6] A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: a scalable and flexible data center network. In ACM SIGCOMM Computer Communication Review, volume 39 of SIGCOMM 09, pages ACM, Acm, [7] A. Gupta, J. Kleinberg, A. Kumar, R. Rastogi, and B. Yener. Provisioning a virtual private network. In Proc. ACM STOC, [8] D. Xie, N. Ding, Y. C. Hu, and R. Kompella. The only constant is change: Incorporating time-varying network reservations in data centers. In Proc. ACM SIGCOMM, pages , Matthias Rost (TU Berlin) Revisiting Virtual Cluster Embeddings Paris, September