How To Connect Two Servers Together In A Data Center Network

Transcription

1 DPillar: Scalable Dual-Port Server Interconnection for Data Center Networks Yong Liao ECE Department University of Massachusetts Amherst, MA 3, USA Dong Yin Automation Department Northwestern Polytech University Xi an, ShanXi 77, China Lixin Gao ECE Department University of Massachusetts Amherst, MA 3, USA Abstract Data centers are becoming increasingly important infrastructures for many essential applications such as ata intensive computing an large-scale network services. A typical future ata center can consist of up to hunres of thousans of servers. Yet, the conventional ata center networks are not able to keep up with the network banwith requirement for efficiently connecting that huge number of servers in a cost-efficient manner. In this paper, we present DPillar, a highly scalable ata center interconnection architecture, which uses only low-en off-theshelf commoity PC servers an switches. DPillar has minimal requirements for the equipment. The switches are low-cost plugan-play layer- evices an servers are ual-port commoity PCs. The salient feature of DPillar is that it expans to any number of servers without requiring to physically upgrae the existing servers. We present simple yet efficient routing schemes for DPillar an evaluate the routing performance via simulations. I. INTRODUCTION Data centers with a cluster of commoity servers become common places for ata storage, ata analysis, an running large-scale network services [1,]. Such a ata center infrastructure is riven by the eman of petabyte of ata storage an high computation power require for processing the ata. It is commonly projecte that the eman for ata storage an processing will grow rapily as more ata are available for applications such as web searching, meical image processing, social network mining, an scientific computing. To meet the eman of the growth, one of the essential requirements for ata center infrastructure is that it must scale to hunres of thousans or millions of servers. While inexpensive commoity PCs make it possible to expan a ata center to millions of servers, interconnecting these servers in a scalable an cost-efficient fashion can be challenging. With a ata center of increasing server number an storage size, the communication banwith has to scale more than linearly (or squarely) to meet the banwith eman of frequent ata accessing an shuffling in istribute ata processing an storage. In orer to keep the interconnection cost low, one natural choice for interconnecting these servers is to leverage commoity harware such as inexpensive Ethernet switches an the existing network cars in commoity servers. So far, there are two approaches for interconnecting servers with commoity switches. The first approach is switch centric where the switch functionality is extene to accommoate the nee of the interconnection, while requiring no moification to the servers [3] [5]. The secon approach is server centric where each server acts as both ata processing/storage an ata relay noe while requiring no change to the switches [6] [8]. In this paper, we take a server centric approach an propose a server interconnection structure calle DPillar. Each server in a DPillar network is a computation workstation as well as an intermeiate noe relaying ata between other servers. A server centric esign offers several attractive avantages as compare to a switch centric esign. First, shifting the networking functionalities from a separate switching fabric to the servers provies much higher egree of programming capability, which facilitates the esign of intelligent routing schemes. Secon, a server centric esign is much more costefficient because it uses only low-en layer- ummy switches. The network structure of DPillar resembles existing multistage interconnection schemes [9,]. owever, DPillar offers several practical avantages in builing large size an scalable ata centers. DPillar aims to leverage plug-an-play commoity Ethernet switches with only layer- switching capability. Ethernet switches with moerate number of ports (e.g., 4 or 48 ports) an with the ability to switch at line spee are wiely available an relatively inexpensive. Layer- Ethernet switches also have the avantage of requiring minimal configuration effort as they are basically plug-an-play evices. Further, DPillar requires only two network interfaces for each server. As most off-the-shelf PC servers an servers in existing ata centers alreay have two high-spee Gbit Ethernet ports, one primary port an one backup port, there is no nee to physically upgrae the servers when using new commoity servers or reusing servers in existing ata centers to buil a DPillar ata center. When expaning an existing DPillar network with aitional servers, it is not require to upgrae existing servers in the ata center either (note that although upgraing servers like installing aitional NICs is cheap in terms of the equipment cost, but the time an human power neee to upgrae tens or hunres of thousans servers are very expensive). Therefore, DPillar can scale to any number of servers with minimal eployment overhea. Despite the fact that each server has only two network interfaces, DPillar offers rich connections between servers an the aggregate banwith can facilitate ata-intensive applications. A DPillar network structure is totally symmetric, so that it removes any network bottleneck at the architecture level. We have esigne a simple yet highly efficient routing scheme

2 for DPillar network. One salient feature of the propose routing scheme is that it eliminates the nee of oing table lookup in packet forwaring. Our prototyping implementation using commoity PC shows that the PC servers can perform ata forwaring in line spee without consuming significant resources at the servers [11]. Therefore, such an interconnection structure is feasible for the server centric approach. Furthermore, we propose routing schemes that can efficiently hanle a wie range of failures in DPillar network. The rest of this paper is organize as follows. Section II presents the topological structure of the DPillar network. Section III an section IV are evote to the iscussion of routing in DPillar network. Section V presents the performance evaluation of DPillar. Section VI conclues this paper. II. INTERCONNECTION OF DPILLAR In this section, we first present the interconnection structure of DPillar. Then we iscuss the topological properties of DPillar an the cost of builing such a network. We also iscuss the ifferences between DPillar an a closely relate multi-stage interconnection network. A. DPillar Network Structure A DPillar network is built of two kins of evices, ual-port servers an n-port switches. The servers are arrange into k columns an so are the switches. ence, a DPillar network is uniquely efine by two parameters, n an k. We call such a DPillar network an (n, k) DPillar network. We use k 1 to represent the k server columns an S S k 1 to represent the k switch columns. The k server columns an k switch columns are alternately place along a cycle, as shown in Fig. 1. Visually, it looks like thek columns of servers an switches are attache to the cylinrical surface of a pillar. Using its two ports, a server in each server column is connecte to two switches in its two neighboring switch columns. In other wors, for a server in column i, one of its ports is connecte to a switch in column S i an the other port is connecte to a switch in column S (i+k 1)%k. For a switch in column S i, half of its n ports are connecte to n/ servers in i an the other half are connecte to n/ servers in (i+1)%k. For easy escription, in the rest we call server column (i+1)%k a clockwise neighboring column of i an (i+k 1)%k a counter-clockwise neighboring column of i.... S 1 Si -1 1 counter- clockwise clockwise S i Si... k-1 Sk-1 Fig. 1. The vertical view of a DPillar network. In a DPillar network with k columns of servers, each server column has (n/) k servers; each switch column has (n/) k 1 switches. For the(n/) k servers in any server column i, each of them is assigne with a unique k-symbol label (ν k 1...ν ), where ν i [,n/ 1] ( i k 1). Uner this naming scheme, one server in DPillar can be uniquely ientifie as (C,ν k 1...ν ), which means a server with label (ν k 1...ν ) in server column C. We call (C,ν k 1...ν ) the ID or the aress of the server. Given the IDs of the servers in a DPillar network, the interconnection between the servers an the switches is as follows. For all the (n/) k servers in any server column C an its clockwise neighboring server column (C+1)%k, they can be ivie into (n/) k 1 groups, with each group having n servers. The labels of the n servers in the same group have the following property. That is, their labels are the same if the Cth symbol (i.e., symbol ν C ) is remove. It is easy to see that among the n servers within the same group, half of them are from C an the other half are from (C+1)%k. The n servers in the same group are connecte to the same switch in switch column S C. In other wors, given any label (ν k 1...ν C...ν ), there are n/ servers in C whose labels are (ν k 1...ν C...ν ) where ν C n/ 1; there are n/ such servers in (C+1)%k too. Those n servers are connecte to the same n-port switch in S C. label Fig S 1 S Two-imension view of a DPillar network. Fig. shows an (8,) DPillar network (note that we uplicate server column ). Each server column has ( 8 ) = 16 servers. The label of each server has two symbols. The first row of servers in Fig. have label () an the last row of servers have label(33). If we select a label(ν 1 ν )=(), there are four servers in 1 whose labels are (ν 1 ) with ν 1 3, i.e., (), (), (), an (3). There are four servers in whose labels are (), (), (), an (3) too. Those eight servers are connecte to the same switch in switch column S 1. B. DPillar Topological Properties After presenting the interconnection of DPillar, we procee to stuy the basic topological properties of DPillar. 1) Number of servers: Since each server column has (n/) k servers, there are k(n/) k servers in an (n,k) DPillar network. Let N represent the total number of servers in an (n,k) DPillar network, we have N = k(n/) k. Consiering that 48-port Gbit Ethernet switches are wiely available now an relatively inexpensive, a (48, 3) DPillar network has 41,47 servers. The number of servers will be about 1.3 million for a (48,4) DPillar network. If we buil a (48, 5) DPillar network, it has about 4 million servers. ) Number of switches: As we have mentione, each switch column has (n/) k 1 switches in an (n,k) DPillar network. As there are k switch columns, the total number

3 of switches is k(n/) k 1. Actually, there is an explanation why there are (n/) k 1 switches in each switch column. If we change all other symbols in label (ν k 1...ν C...ν ) except symbol ν C, there are (n/) k 1 ifferent combinations. Each of those (n/) k 1 combinations requires one switch to connect n servers whose labels are (ν k 1...ν C...ν ) where ν C n/ 1. Therefore, the number of switches in each switch column is (n/) k 1 an the total number of switches in an (n,k) DPillar network is k(n/) k 1. 3) Bisection with: Bisection with is an important criterion to quantify the performance of an inter-connection network. It is efine as the smallest number of eges removal of which ivies the noes in the network into two parts of equal size. Larger bisection with means the network can sustain more communications between noes, which is important for running communication intensive applications like MapReuce [1]. The bisection with of an (n, k) DPillar network is (n/) k, as state in Proposition II.1. The proof is presente in [11]. Proposition II.1 The bisection with of an (n, k) DPillar network is close to (n/) k. C. Builing Cost DPillar network is cost-efficient as it uses inexpensive commoity harware. ere we provie some example bugets of builing DPillar networks. We ignore the cost of servers an focus on the networking evices (switches an Ethernet cables). As most off-the-shelf servers alreay integrate ualport interfaces, there is no nee to invest on NICs. TABLE I shows the total cost an the per-server cost of builing DPillar networks with four servers columns, when ifferent types of switches are use. The prices for the four types of switches are gotten from an online retailing store ( an we assume each cable costs $1. We expect their wholesale price woul be even lower. In general, letting U s be the unit price of a n-port switch an U c be the unit price of an Ethernet cable, the average cost of connecting one server in the DPillar network is (U s /n+u c ). TABLE I TE COST OF TE NETWORKING DEVICES WEN USING FOUR TYPES OF SWITCES TO BUILD DPILLAR NETWORKS WIT FOUR SERVER COLUMNS. switch type 8-port 16-port 4-port 48-port switch unit price $5 $15 $18 $6 # of servers 1,4 16,384 8,944 1,37,4 networking cost $14,848 $339,968 $1,4,48 $35,831,88 per-server cost $14.5 $.75 $17 $7 D. Contrasting DPillar an Existing Network Structure Our DPillar network is closely relate to wrappe butterfly network [9], which is one of the multi-stage interconnection networks extensively stuie before. owever, the servers in a wrappe butterfly network must have four ports. Because most commoity servers an servers in existing ata centers integrate only two ports, we have to physically upgrae the servers if using a wrappe butterfly network to interconnect them. Although the cost of investing aitional network interfaces is not an issue, installing those interfaces in a large number of servers can be quite time an manpower consuming. By connecting the servers via switches, DPillar can use offthe-shelf an existing ual-port servers to buil a scalable ata center network. The bisection with or an (n,k) DPillar network is equal to the number of servers in each server column. A wrappe butterfly network also has this property [9]. owever, a wrappe butterfly network uses egree 4 noes, but DPillar achieves the same bisection with using egree noes. III. ROUTING IN DPILLAR NETWORK Because of its symmetric structure, routing in DPillar network can be simple an efficient. In this section, we first escribe the packet forwaring process in DPillar, base on which we esign an efficient DPillar routing algorithm. We also briefly iscuss the performance of the routing algorithm. A. Two-Phase Packet Forwaring in DPillar The packet forwaring process in DPillar can be ivie into two phases. In the first phase, the packet is forware from the source server to an intermeiate server whose label is the same as the estination s label. In the secon phase, the packet is sent from that intermeiate server to the estination. We consier a source server s sens a packet to estination server. The aresses of those two servers are (C s,l s ) an (C,L ), where L s an L are the k-symbol labels of s an, i.e., L s =(ν k 1 s...ν s), L =(ν k 1...ν ), an L s L. 1) Phase one helix phase: From server s in column Cs, the packet can be sent to a server s 1 in column (Cs+1)%k. The labels of s an s 1 are the same except the C s th symbol of s 1 s label can be any number in [,n/ 1]. If s 1 sens the packet to s in column (Cs+)%k, the labels of s 1 an s s are the same except that the ((C s + 1)%k)th symbol of s s label can be any number in [,n/ 1]. We see that forwaring a packet one hop can change one symbol in the label of the server receiving that packet. When a packet is always forware from one server column to its clockwise neighboring server column, the packet can reach a server with any given label in k hops. For example, in an (n,k) DPillar network, the trace of a packet forware from (,...) to (k 1,1...1) is (,...) (1,...1) (,...11) (k 1,1...1). As this path resembles a helix, we call this packet forwaring phase the helix phase. Note that in the helix phase, we can sen the packet to either a server in the clockwise neighboring column or a server in the counter-clockwise neighboring column. owever, the irection of forwaring a packet shoul not be change back an forth in orer to avoi loops. Some fiel in the packet heaer can be use to recor the forwaring irection information of this packet. In DPillar routing, the efault irection of forwaring a packet in the helix phase is the clockwise irection. ) Phase two ring phase: After the packet is forware to server, whose label is the same as the label of estination, one can forwar the packet to by always sening it to the server in the clockwise neighboring column whose label is L too, or sening along the counter-clockwise irection. We select the shorter one among those two paths in our DPillar routing. In other wors, suppose server is in column C,

4 sens the packet to a server in column (C +1)%k if(c + k C )%k k ; otherwise server sens the packet to a server in column (C +k 1)%k. In either case, the label of the nexthop server shoul be L. As the trace of the packet forwaring in this phase is like a segment in a ring, we call it the ring phase. B. Routing Algorithm Algorithm 1 shows the pseuocoe of the routing algorithm. This algorithm takes the aress of the server running this algorithm (C s,l s ), the estination server s aress (C,L ), an the forwaring irection D as input parameters. D=1 means the irection is clockwise;d= 1 inicates the counterclockwise irection. The efault value of D shoul be 1. The output is the aress of the nexthop server (C u,l u ). Algorithm 1: SRoute(C s,l s,c,l,d) input : (C s,l s) is the aress of the server running this algorithm. (C,L ) is the estination. D is the forwaring irection recore in the packet heaer (either 1 or 1). L s=(νs k 1...νs), L =(ν k 1...ν). output : Aress of the nexthop server (C u,l u). /* L s νs Cs means removing the C sth symbol from label L s */ 1 if {(C +k-c s)%k 1 an L s-νs Cs == L -ν Cs } or {(C s+k-c )%k 1 an L -ν C ==Ls-νC s } then C u C ; L u L ; 3 else /* s cannot irectly reach */ 4 if L s == L then /* the ring phase */ 5 L u L ; 6 if (C +k C s)%k k then Cu (Cs +1)%k; 7 else C u (C s +k 1)%k; 8 else /* the helix phase */ 9 L u (νs k 1...ν...ν s s); C u (C s +D +k)%k; return (C u,l u); The DPillar routing presente in Algorithm 1 oes not compute the shortest path between two servers. owever, the paths compute by Algorithm 1 have boune length. ere we stuy what is the length of the longest path in a DPillar network. In counting the path length, we treat the istance between two servers connecting to the same switch as one hop, because the switches are ummy layer- connection meia. From source server s, a packet nees to be forware at most k times to reach a server, whose label is the same as estination server s label. After that, from server, the packet still nees to be forware at most k/ hops to reach server. Therefore, in an (n, k) DPillar network, the longest path compute by Algorithm 1 is k + k/. IV. ANDLING FAILURES IN DPILLAR The rich connections in DPillar facilitate the esign of simple yet efficient fault-tolerant routing scheme. In this section, we present the esign of a fault-tolerant routing algorithm which can bypass a wie range of failures in DPillar. A. Discovering Failures To bypass failures in DPillar network, the first step woul be etecting the failures. Each server in DPillar runs a lightweight ello protocol to report the reachability to other servers connecte to the same switches. If a server a oes not hear ello message from server b for a certain perio of time, a assumes b is not irectly reachable. Servers shoul not forwar any ello messages. After iscovering the failures, the next question woul be how to bypass those failures. In the following, we iscuss how to bypass failures when the packet forwaring is in the helix phase an the ring phase, respectively. B. Bypassing Failures in elix Phase In bypassing failures in the helix phase, our goal is to etour a packet so that it can still reach a server whose label is the same as the estination of that packet. After that, the packet is forware in the ring phase an section IV-C iscusses how to bypass failures in the ring phase. Our fault-tolerant routing scheme works in the following three steps in the helix phase: (1) if a nexthop server in the clockwise neighboring column is not reachable, we first try to sen the packet to another irectly reachable server in the clockwise neighboring column; () if none of the servers in the clockwise neighboring column is irectly reachable, the packet takes a u-turn an be forware to a server in the counter-clockwise neighboring column; (3) after the u-turn, the packet shoul be forware along the counter-clockwise irection throughout the helix phase. label Fig. 3. A (4, 3) DPillar network example. 1) Bypassing a faile clockwise server: Suppose server s sens a packet to estination. Accoring to Algorithm 1, s shoul sen the packet to nexthop u. If s fins that u is not irectly reachable any longer, s shoul try to bypass the faile server u. owever, simply forwaring the packet to an alternative clockwise nexthop can cause loops in some scenarios. For example, in Fig. 3, when server (,) sens a packet to (,1), Algorithm 1 tells us the ID of the nexthop shoul be (1,1). If server (1,1) has faile an server (,) tries to bypass that failure by sening the packet to server (1,), the path taken by the packet will be (,) (1,) (,) (,), which is a loop. The reason for the loop to occur is that, server s tries to bypass u an sens the packet to server v, but s is in the path (compute by Algorithm 1) from v to a server whose label is the same as estination. To avoi this loop, we can let v sen the packet to a clockwise neighbor w, so that neither s nor u is in the path from w to a server whose label is the same as estination. For example, if (1,) forwars the packet to (,), the path taken by the packet will be (,) (1,) (,) (,) (1,11) (,1)

5 We use tunneling to ensure the packet is forware from s to w (they are two hops away) without using the faile server u. In the above example, server (,) encapsulates the packet with another heaer, whose estination is set to (,). Tunneling the packet between (,) an (,) makes sure the path of the encapsulate packet is (,) (1,) (,). Therefore, the faile server (1,1) is bypasse. We conclue the iscussion into Proposition IV.1. Note that we omit the %k in presenting the inexes for clarity. Proposition IV.1 When sening a packet to estination (C,ν k 1...ν ), server (C s,νs k 1...νs) can bypass server (C s + 1,νs k 1...ν Cs...ν s) by tunneling the packet to (C s +,νs k 1...νw Cs+1 νw Cs...νs), if νw Cs ν Cs an νw Cs+1 νs Cs+1. Proof: Due to the symmetry of DPillar, without losing generality, we consier servers(,νs k 1...νs), bypasses server (1,νs k 1...νsν 1 ) in reaching estination (C,ν k 1...ν ). Server s tunnels the packet to server w, whose aress is (,νs k 1...νwν 1 w), where νw ν an ν1 w νs 1. To tunnel the packet from s to w, the nexthop is (1,νs k 1...νsν 1 w). Because νw ν, that nexthop is not the faile server (1,νs k 1...νsν 1 ). After the packet is forware from w to server s in column, the aress of s is (,ν k 1...νwν 1 w). As νw ν 1 s 1, s is not s. Server s forwars the packet to server s 1 in column 1 whose aress is (,ν k 1...νwν 1 ). Because ν1 w νs 1, s 1 is not the faile server (1,νs k 1...νsν 1 ). After s 1 forwars the packet, it reaches a server in whose label is the same as the estination. ) Making a packet u-turn : Proposition IV.1 assumes a server can always forwar a packet to another server in its clockwise neighboring column in the helix phase. owever, it is possible that server s cannot sen the packet to any servers in its clockwise neighboring column, e.g., the server (,) in Fig. 3 has a link failure so it is isconnecte from the top switch between an 1, or the top switch between an 1 has faile. In that case, server s nees to change the packet forwaring irection (make a u-turn ) to bypass the failure, i.e., if a packet cannot be forware to a server in the clockwise irection, it shoul be forware to a server in the counter-clockwise neighboring column. The forwaring irection information shoul be recore in the packet heaer, so that other servers will forwar this packet along the new irection. Similar to the rationale behin Proposition IV.1, we can prove the following Proposition IV.. Again, the %k is omitte in presenting the inexes for clarity. Proposition IV. When sening a packet to estination (C,ν k 1...ν ), if server (C s,νs k 1...νs) cannot reach any server in Cs+1, it can bypass the failure by changing the packet forwaring irection an sening the packet to server (C s 1, νs k 1...νw Cs 1...νs), where νw Cs 1 νs Cs 1. Proof: We consier that server s, whose ID is (,νs k 1...νs), sens a packet to estination (C,ν k 1...ν ) but s cannot reach any server in 1. To bypass the failure, server s changes the forwaring irection of the packet an sens it to nexthop w, whose ID is (k 1,νw k 1...νs), where νw k 1 νs k 1. After the packet is forware along the counter-clockwise irection from w to some server s 1 in 1, the aress of s 1 is (1,νw k 1...ν ν1 ν s). Because νw k 1 νs k 1, s 1 is not connecte to server s by the same switch. After s 1 forwars the packet to some server s in, s s aress is (,νw k 1...ν ). Because νw k 1 νs k 1, s is not s so no loop occurs. After s forwars the packet to server s k 1 in column k 1, the packet reaches a server whose label is the same as estination server. 3) Bypassing a faile counter-clockwise server: Once the forwaring irection of a packet is change, it is recore in the packet heaer an all servers shoul forwar the packet along the new irection. To avoi potential forwaring loops, if the forwaring irection of a packet was change before, we shoul not change its forwaring irection again. Suppose server s selects a counter-clockwise neighbor u as the nexthop accoring to Algorithm 1, if u is unreachable an there are other reachable counter-clockwise neighbors, s can bypass the failure accoring to the following Proposition IV.3. For a packet whose irection was change from clockwise to counter-clockwise an it cannot be forware along the new irection (no server in the counter-clockwise neighboring column is irectly reachable), the packet shoul be roppe. Proposition IV.3 When sening a packet to estination (C,ν k 1...ν ), server (C s,νs k 1...νs) can bypass server (C s 1,νs k 1...ν Cs 1...νs) by tunneling the packet to (C s,νs k 1...νw Cs 1 νw Cs...νs), if νw Cs 1 ν Cs 1 an ν Cs w νs Cs. The proof of Proposition IV.3 is similar to the proof of Proposition IV.1. We omit it here to save space. C. Bypassing Failures in Ring Phase Bypassing a faile server in the ring phase is relatively straightforwar. As there are two ways to forwar a packet to the estination in the ring phase, the clockwise irection or the counter-clockwise irection, if it cannot be forware along one irection, a packet shoul be forware along the other irection. To avoi forwaring loops, the forwaring irection of a packet shoul not be change more than once. If a packet has change its forwaring irection in the ring phase an it encounters a faile server again, we rop that packet. D. Fault-tolerant Routing Algorithm Base on the above iscussions, we esign the DPillar fault tolerant routing algorithm as shown in Algorithm. FTRoute first calls SRoute to compute a nexthop (C w,l w ). Then FTRoute tests whether (C w,l w ) is reachable. If it is, FTRoute oes nothing; otherwise it tries to fin a new nexthop accoring to the basic ieas specifie in Proposition IV.1 IV.3. If the new nexthop is null, the packet shoul be roppe to prevent forwaring loops. V. EVALUATIONS In this section, we stuy the macroscopic behavior of DPillar by simulating the packet routing an forwaring in a large scale DPillar network, using a simulation tool evelope by ourselves. Please refer to [11] for the microscopic experiments in measuring the forwaring performance of DPillar.

6 Algorithm : FTRoute(C s,l s,c,l,d) input : (C s,l s) is the aress of the server running this algorithm. (C,L ) is the aress of the estination. D is the forwaring irection, either 1 or -1. output : Aress of the nexthop server (C w,l w). (C w,l w) = SRoute(C s,l s,c,l,d); if (C w,l w) is reachable then return (C w,l w); if L s==l then /* the ring phase */ if the packet irection was change then C w null; L w null; 6 else 7 if (C s +k C )%k k then Cw (Cs +1)%k; 8 else C w (C s +k 1)%k; 9 else /* the helix phase */ if the packet irection was change then 11 apply P roposition IV.3 to get new nexthop; 1 if nexthop exists then set C w an L w; 13 else C w null; L w null; else apply P roposition IV.1 to get new nexthop; if nexthop exists then set C w an L w; else change packet irection accoring to P roposition IV.; if nexthop exists then set C w an L w; else C w null; L w null; return (C w,l w); Average path length: In our simulations, we focus on a scenario where the DPillar network is built from 1-port switches, i.e., n = 1, an the number of server columns k varies. For each network, we ranomly select, sourceestination pairs an simulate packet routing an forwaring in the network. Fig. 4 plots the results of the average path lengths. As expecte, the average path length is proportional to the number of server columns in a DPillar network. For all server column k, the simulate average path length is always about % shorter than the maximum path length k+ k/. path length k: number of server columns Fig. 4. Average path length in DPillar routing. Average path length in failure-tolerant routing: We also stuy the fault-tolerant behavior of DPillar routing. In this simulation, we use a (1,4) DPillar network to conuct our experiments. In each simulation instance, we first ranomly fail a certain number of servers. Then we ranomly select, source-estination pairs an simulate the fault-tolerant routing scheme in Algorithm. We range the number of faile servers from 1 to 3 an plot the average path length as a function of the number of faile hosts. Our simulation results show that the average path length increases as there are more faile servers in the DPillar path length number of faile servers Fig. 5. Path length in fault-tolerant routing. network. owever, no packet rop occurs in our simulations, even when 3 servers in the DPillar network have faile. As a (1, 4) DPillar network has about 5, servers, 3 server failures mean 6% of the servers are faile. VI. CONCLUSION In this paper, we present DPillar, a scalable ata center network architecture which uses only commoity off-the-shelf harware. DPillar can easily scale to huge number of servers without imposing any aitional requirements to the evices, such as installing aitional NICs in the servers. The topology of DPillar is totally symmetric an a DPillar network has balance network capacity. DPillar is server centric an the networking intelligence is place in the servers. The switches use in DPillar are merely ummy layer- evices connecting the servers. We esigne simple yet efficient routing schemes for DPillar. Our experiment stuies show that the routing schemes are high-performance an efficient in bypassing failures in the network. ACKNOWLEDGMENT This work is partially supporte by NSF grants CNS an CNS Dong Yin was a visiting stuent at UMass, supporte by China State Scholarship Fun CSC-8698, when this work was performe. REFERENCES [1] Amazon elastic compute clou, [] L. Rabbe, Powering the Yahoo! network, Nov. 6. [Online]. Available: [3] M. Al-Fares, A. Loukissas, an A. Vahat, A scalable, commoity ata center network architecture, in Proceeings of SIGCOMM 8, 8. [4] R. N. Mysore an et al., Portlan: A scalable fault-tolerant layer ata center network fabric, in Proceeings of SIGCOMM 9, 9. [5] Cisco ata center infrastructure.5 esign guie, Dec. 7. [Online]. Available: ns7/c649/ccmigration 9186a pf [6] C. Guo,. Wu, K. Tan, L. Shi, Y. Zhang, an S. Lu, Dcell: a scalable an fault-tolerant network structure for ata centers, in Proceeings of SIGCOMM 8, 8. [7] D. Li, C. Guo,. Wu, K. Tan, Y. Zhang, an S. Lu, FiConn: Using backup port for server interconnection in ata centers, in Proceeings of INFOCOM 9, 9. [8] C. Guo, G. Lu, D. Li,. Wu, X. Zhang, Y. Shi, T. Chen, Y. Zhang, an S. Lu, BCube: A high performance, server-centric network architecture for moular ata centers, in Proceeings of SIGCOMM 9, 9. [9] F. T. Leighton, Introuction to Parallel Algorithms an Architectures: Arrays. Trees. ypercubes. Morgan Kaufmann, 199. [] M. Jeng an. Siegel, A fault-tolerant multistage interconnection network for multiprocessor systems using ynamic reunancy, in Proceeings of ICDCS 86, [11] Y. Liao, D. Yin, an L. Gao, Dpillar: Scalable ual-port server interconnection for ata center networks, Tech. Rep., 9. [Online]. Available: [1] J. Dean an S. Ghemawat, MapReuce: Simplifie ata processing on large clusters, in Proceeings OSDI 4, 4.