Optimal Data Distribution for Heterogeneous Parallel Storage Servers Streaming Media Files

Transcription

1 Optimal Data Distribution or Heterogeneous Parallel Storage Servers Streaming Meia Files Christian Ortol Department o Computer Science, University o Freiburg, Georges-Koehler-Allee 5, 79 Freiburg, Germany ortol@inormatik.uni-reiburg.e Christian Schinelhauer Department o Computer Science, University o Freiburg, Georges-Koehler-Allee 5, 79 Freiburg, Germany schinel@inormatik.uni-reiburg.e Abstract We consier the problem o istributing meia iles or streaming on a istribute storage network, where servers have heterogeneous capacities an banwiths. Regaring networking the servers banwiths are the bottlenecks or streaming. We present an algorithm that computes an assignment o n iles to m servers or istributing meia iles such that the streaming spee requirements an capacity constraints are kept. As an aitional eature this assignment algorithm works on, i.e. it can assign each ile without iles to be store later on. Our algorithm computes the ata assignment in O(nm+m log m) outperorming ar program solvers. I. INTRODUCTION Streaming meia iles has become a stanar application or PCs an smart TVs in moern homes. Currently, this requires a eicate streaming server powerul enough to provie the banwith or high einition mulia iles an at the same have enough storage capacity to contain a large meia library. Distribute ile systems coul rener such servers unnecessary by accumulating the require resources rom multiple potentially less powerul evices like noes in a boy area network (BAN), which may be too slow or equippe with too little memory or inepenently serving the meia iles alone. Distribute ile systems have the avantage that they can be built to scale or many large iles or access by many clients in parallel or minimize parallel rea o a ile. None o these optimizations solve the problem pose by the combination o banwith requirement with a restricte storage on evices with heterogeneous banwith capabilities. Here, we solve the istribution o ile ragments across a heterogeneous network o ile servers. This requires a partition o a meia ile, an the assignment o the parts to meia servers, which on eman serve their parts with a ixe known bit rate. The necessity o meia iles istribution comes rom resource restricte storage servers with limite storage capacity an storage access rates. Our objective is to partition an assign the meia iles to the servers, either o or on. On means that the iles arrive in a ranom orer an nee to be assigne once or all. In the o case, we know the sizes an streaming rates o all iles to come an want to compute an optimal strategy. In a homogenous setting it is obvious that a ile can just be partitionento equal parts an place on all server with same size an spee. But with servers varying in storage capacity an banwith, an equal istribution may clog up high banwith or low storage servers very quickly. We show in this paper how to achieve an optimal istribution on. Thereore we can solve this ile istribution problem without incurring any cost or reistributing iles later on. This perect istribution o ata in an inocommunication system can be use as a tool to eiciently hanle ata or representation-briging communication an meiastreaming, e.g. sensory ata can be istribute an storen a BAN to later be playe back to its user. Relate Work.: Since the introuction o reunant arrays o inepenent iscs in the seminal paper by Patterson et al. [7] istribute storage has evolve. While RAID systems o not know heterogeneous isc sizes, network connections, hotspot iles, scalability an optimizing energy consumption, moern istribute storage systems consier such actors. The Google File System (GFS) [] was esigne to satisy the nee or a highly scalable an high throughput system or Google s web search service. It uses a large number o commoity computers (chunk servers) combine with a single controlling server in a homogenous network setting. Files are split into chunks an istribute to the chunk servers. For ile retrieval a client queries the location o the chunks rom the master server an ownloas the chunk irectly rom the chunk server. Parallelization comes into play when multiple clients access ierent chunk servers an hot spot chunks are copie to multiple chunk servers. Dynamo [] was evelope or Amazon an ocuses on parallel write operations an high availability. Instea o a central master server, it uses consistent hashing [] to istribute ata onto the servers. By utilizing virtual copies o noes in the consistent hashing it can hanle servers with heterogeneous storage capacities. Like GFS it oes not hanle heterogeneous banwiths as all servers are assume to be in the same local network. Various other istribute storage systems utilize peer-topeer networks [], [5], []. These works ocus on ierent challenges like the reputation o peers, the reliability o store iles with oten isconnecte peers or proviing byzantine ault tolerance in the network. The assumption o an homogenous unerlying network inrastructure restricts the applicability o these systems. Storage inrastructures ten to evolve over an become heterogeneous, which stanar solutions o not cover. Likewise any Internet base service will inevitably ace a large variety o ierent connections.

2 The Distribute Parallel File System (DPFS) by Shen an Chouhary [] was built to hanle such an heterogeneous networking inrastructure. Files to be store are broken up into chunks an istribute among the network o chunk servers. DPFS ocuses on how the chunks shoul be istribute or optimal perormance uner expecte access pattern, but they also accommoate or ierent banwiths by istributing chunks proportionally to a given perormance actor among servers using a simple greey algorithm. While consistent hashing can be aapte with the help o virtual servers to heterogeneous settings, Distribute Heterogeneous Hash Tables (DHHT) [8] are alreay esigne or heterogeneity. Servers can be weighte accoring to capacity or connection banwith an receive a larger number o chunks. The DHHT was later utilizen a istribute storage system in [9]. There an optimal solution or the average parallel transer o a ocument is shown. In the work by Langner et al. [6] the uploa o chunks in a setting with asymmetric connections is consiere. They show a solution that also minimizes the parallel or uploa an ownloa. The moel presenten this paper is base on the master thesis o Schott []. He presents a similar, but less eicient algorithm. II. THE PROBLEM SETTING AND CONTRIBUTION We consier a ixe set o m servers,..., s m with storage capacities c,..., c m measuren bits. Let b i enote the banwith o a server, i.e. the number o bits a storage server can stream per secon. Now, the meia iles,..., n are ae to the storage servers, where j enotes the size o ile j an j the necessary streaming ata rate o this ile. We assume that the storage server is the bottleneck. So, the or ownloaing the ile i is etermine by the slowest part. Now the ile is assigne to the servers inicate by the assignment matrix A = (a i,j ) i [n],j [m with a i,j inicates the number o bits o ile i store on storage server s j. Clearly, all bits o the ile nee to be istribute on all storage servers. m a i,j = i or all i [n]. () j= O course, the capacity o the server cannot be exceee. n a i,j c j or all j [m]. () i= We assume that the banwith o the storage server is the bottleneck, while the network oes not pose urther constraints. From the minimum ata rate o a ile i it is clear that the maximum it may take to retrieve a ile is i /. For each server s j the minimum to sents parts o i is a i,j /. For continuous streaming we nee a i,j / i / or each assignment. This leas to the main constraint o the parallel heterogeneous streaming problem: O course, we also nee to consier where to place each bit o ata. For this, the ile nees to be partitionento blocks o aequate size resulting rom a ar selection o the ata. There is a straight-orwar solution or this problem an thereore we o not consier it in this paper. Here, we concentrate only on the assignment problem. Deinition The parallel, heterogeneous o streaming assignment problem is, given servers,..., s m an all iles,..., n, compute an assignment A which satisies constraints (), () an (). This is clearly a ar program. We solve this problem in O(nm + m log m), which is aster than any existing ar program solver. For the on streaming assignment problem servers,..., s m are given at the beginning. The iles,..., n are given sequentially, such that the assignment a i,j or ile i must be compute beore iles i+,..., n are given. The task is to compute an assignment which satisies constraints (), () an () i possible. We call an on streaming assignment perect, i it can assign the same number o iles as an o assignment. Our algorithm is an on algorithm, an thus perect. III. AN EFFICIENT OFFLINE AND ONLINE SOLUTION The key to our solution is the notion o sustainability. Deinition The sustainability σ j o a storage server s j is eine as the it nees to rea out all ata, i.e. σ j := c j /. Servers with equal sustainability can be split an joine without aecting the solvability o the problem. Lemma For any set o iles the assignment problem onto servers S = {,..., s m }, where σ m = σ m can be solve an only i the assignment problem to servers S = {,..., s m, s } can be solve, where the capacity o s is c m + c m an the banwith o s is b = b m + b m. Proo: Assume that (a i,j ) i [n],j [m] is an assignment o the iles to S. Then the assignment (a i,j ) i [n],j [m] to S is vali where { a ai,j, j [m ] i,j = a i,m + a i,m, j = m Clearly, constraints () an () hol. For constraint () we observe a i,m + a i,m i b m + b m = i b. Now assume that an assignment (a i,j ) i [n],j [m] or S is given, then an assignment or S can be compute as ollows, j [m ] a i,j = a i,j a i,m b m b m + b m, j = m a i,j i or all i [n], j [m]. () a i,m b m b m + b m, j = m

3 Again constraints () an () are straight-orwar. For constraint () we have b m a i,m = a i,m b m + b m an analogously or a i,m. i b m The irst step to solve the assignment problem is to sort all storage servers with respect to their sustainability σ i, such that σ σ... or storage servers,..., s m. Let t i = i / enote the play o ile i. In real-worl applications iles are continuously ae or remove rom storage server systems. Then, it is costly to reistribute an recompute all assignments when a new ile arrives. So, we consier the case where assignments are mae or each ile separately without the knowlege o uture iles. Hence, rearranging the set o iles, e.g. accoring to their play is not allowe. One might think that this reuces the possible solution space. However, we present with Algorithm an on algorithm which also computes vali assignments or the o problem. So, this algorithm reveals a structural property o the assignment problem. The algorithm sorts all servers accoring to their sustainability. Then it computes or each ile i an assignment in the or-loop between s 7 an 5. The basic iea is a that moves over the storage servers epicten Fig.. The vertical moves continuously rom the right to the let. When it moves over the rectangle representing a server the area to the right o the inicates the assigne amount o ata. However, i it has reache an area o size i /, which correspons to istance t i ater touching a rectangle or the irst, it will stop the assignment to this server because o constraint (). The stops, i the has collecte area o size i or it reaches the vertical axis. In the irst case assignment o the ile can be compute, in the secon case there is no assignment. An eicient implementation o such a technique uses so-calle events. These events are the beginning o rectangles at σ i,j+, the constraint () at σ i,j+ t i, or the halt o the when the complete ile is assigne to the servers in 7. The starts rom the maximum value σ i, an chooses the next event by maximizing over these three cases. Then, the next rectangle will be ae to the assignments in 6. The currently active servers in the while loop are s j,..., s j. Each event nees a special treatment or the next roun, i.e. removing a storage server by increasing j, aing a storage server by increasing j. The stops when all storage is assigne or it runs out o servers j > m. The ollowing notations are useul or analyzing the algorithm. Deine the resiual storage c i,j ater inserting i iles as ollows. c,j := c j or all j [m] c i,j := c i,j a i,j or all i [n], j [m] Algorithm Sweep Line Algorithm or Streaming Assignment Input: storage servers,..., s m with capacity c,..., c m an banwith,..., b m an iles,..., n with ata rates,..., n : Sort storage servers such that σ σ... σ m : or j o m o : σ,j c j / : c,j c j 5: en or 6: σ,m+ 7: or i o n o 8: or j o m o 9: a i,j : en or : t i i / : j : j /* R R {i} */ : s 5: t last σ i, 6: while s < i an j m an t new o 7: t new σ i,j t i 8: i j < m then 9: t new max{t new, σ i,j+} : en : i j j hen{ } : t new max t new, t last i s j j=j : en : i t new then 5: or j j o j o 6: a i,j a i,j + (t last t new ) 7: s s + (t last t new ) 8: en or 9: t last t new : i t new = σ i,j t i then : j j + /* R j R j \ {i} : F j F j {i} */ : else i t new = σ i,j+hen : j j + /* R j R j {i} */ 5: en 6: en 7: or j o m o 8: σ i,j σ i,j a i,j / 9: en or : σ i,m+ : en while : i j > m or t new < then : return File cannot be assigne : en 5: en or 6: return (a i,j ) j [m]

4 The resiual sustainability o the server j ater inserting i iles is enote by σ i,j. σ i,j := c i,j or all i [n], j [m] The assignment o a server measuren play is enote by z i,j. z i,j := a i,j or all i [n], j [m] Note that constraint () translates into the conition z i,j t i or all i, j. In the commentary section the assignments are classiie by the sets R k = {i : < z i,k < t i } or all k [m] F k = {i : z i,k = t i } or all k [m] where F k (ull play) enotes the set o ile inices where the maximum block size is assigne to server s k accoring constraint (). R k (rest o the ile) enotes the set o ile inices with other non-zero assignments o blocks to server s k. I these sets are given, then the assignment can be presente in close orm, see Algorithm. Algorithm Alternative presentation o Algorithm Input: storage servers,..., s m with capacity c,,..., c,m sorte accoring σ j = c j / in escening orer banwith,..., b m an iles,..., n with ata rates,..., n sets R,..., R m, F,..., F m rom Algorithm : or i = o n o : or all j : i F j o : a i,j i : en or j 5: t :i F j t i + j :i R c j i,j i j :i R j 6: or all j : i R j o 7: a i,j c i,j t 8: en or 9: or j o m o : c i,j c i,j a i,j : en or : en or Lemma I Algorithm computes an assignment, then Algorithm computes the same assignment. Proo: I i F j, then z i,j = t i an thereore a i,j / = i /. I i R j, then the rest o the ile is istribute onto the servers j,..., j. For this the position t o the is compute which satisies (σ i,j t ) + t i = i j :i R j j :i F j The assigne storage rom a i,j is then given by (σ i,j t ). We prove the correctness o the algorithm by an inuction over the set o servers. Lemma Algorithm is correct or a single server. Proo: For one server Algorithm has only two kin o events or the : checking or constraint () an checking or constraint (). So, the algorithm ecies correctly, whether the iles can be assigne. Theorem The On Algorithm computes a vali assignment or every set o iles,..., n i it is possible. Thereore, it provies a perect solution to the on problem in O(nm + m log m). Proo: The sorting o all servers requires O(m log m). The sums j j=j usen can be precompute when the interval [j, j ] is change with amortize constant costs. For the run- o the while loop observe that in each roun either j or j is incremente or the loop terminates. Thereore, the inner statements o the loops are perorme m + imes. Thus, assigning each o the n ile takes O(m) steps. We will now prove that or any set o iles,..., n that Algorithm ins a vali assignment i possible. For this we take a closer look when a part o a ile is assigne to a server maximizing constraint (), i.e. i F k, or when it is below but not zero, i.e. i R k. I i F k+, then we have also i F k since σ i,k σ i,k. By this argument, it ollows that i is an element o all the sets F,..., F k. Furthermore, i i R k+hen the assignment to s k is non-zero, i.e. i F k R k. Again i has nonzero assignments or servers,... s k. Furthermore, the has always a non-zero assignment to the irst server, i.e. F R. Lemma states that Algorithm is correct or a single server. We prove the correctness by an inuction over the number o servers an assume that it computes a possible vali assignment or m servers. Assume that an assignment exists an that the algorithm oes not in one. Then, we istinguish two cases. Let n be the number o iles that can be assigne by the algorithm beore it aborts. ) For all i [n ], j [m ] : σ i,j > σ i,j+ : Consier the server s m. So, either σ n,m < or m j= n + < n +. In the later case no vali assignment is possible. The irst case is equivalent to n + i= a i,m > c m, where or all i R m F m (i.e. a i,m ) we have i F... F m an thereore m i a i,m = i j=

5 i. Further- Assume that a i,j Then, constraint () implies a i,j more j [m] a i,j = i an thus, is a vali assignment o all iles. m a i i,m i j= Constraint () implies n i= a i,m c m an thereore i R m F m a i,m c m which implies This contraicts i R m F m i m j= i R m F m a i,m > c m i c m. an thereore in this case Algorithm must in a vali assignment. ) There exists i [n ] an j [m ] : σ i,j = σ i,j+ We show that servers s j an s j+ can be replace by a single server s such that Algorithm chooses the very same assignment, i.e. i a i enotes the assignment o ile i to the new server s then a i = a i,j + a i,j+. Let l := min{i : j [m ] : σ i,j = σ i,j+ }. The capacity o the new server is c j + c j+ an the banwith is b = + c j+ an the sustainability σ = cj+cj+ ++ is thus σ j = c j+ + c j + c j+ + + c j = σ j+. So, the alternative set o servers are sorte as,,..., s j, s, s j+,..., s m accoring to their sustainability. So, server s simply replaces s j an s j+. We consier the ollowing cases an check whether Algorithm chooses the same assignment, i.e. a i = a i,j + a i,j+ without changing all other assignments by an inuction over i. a) i > l: Then, σ i,j = σ i,j+ an thereore i R j i R j+ an F j i F j+. Thereore rom Algorithm an the equivalency o both algorithms it ollows that a i = a i,j + a i,j+ since b = + +. b) i = l: Then i R j an R j+. Let j j an j j + be the participating servers at the event. Then or all l [j, j ]: a i,l = c i,l P = j :i F j b l j :i R j P, where t i + j :i R j c i,j i Since by inuction c i = c i,j + c i,j+ an b = + +, we have a i = a j + a j+. For the other assignments in the interval [j, j ] the values o not change either. c) i < l, i R j R j+ This case is impossible since otherwise σ i,j = σ i,j+ which leas to the contraiction i < l = min{i : σ i,j = σ i,j+ }. ) i < l, i (R j F j ) an (R j+ F j+ ) This case is impossible by the esign o the Algorithm. e) i < l, i R j an F j+ This case is also not possible. ) i < l, i (R j F j ) an (R j+ F j+ ) Then a i = a i,j = a i,j+ =. g) i < l, i (R j F j ) an (R j+ F j+ ) So, the rest o the ile is store on server s j (an alternatively server s ). Then a i,j+ = an a i,j = i t i j :i F j The very same ormula computes a j an thereore a j = a i,j + a i,j+ h) i < l an F j an R j+ I i R j+, then σ i,j+ = σ i,j+ which contraicts the minimality o l. Thereore, j + > m or i R j+ F j+. So, we get a i,j = i a i,j+ = i while a i = i j j = i j j = t i Thereore a i = a i,j + a i,j+. i) i < l, i F j an F j+ Then a i = i bj+bj+ = a i,j + a i,j+. By inuction over i it ollows that all assignments o all servers j < j an j > j + remain the same or both sets o servers. We have assume that an assignment (a i,j ) exists or the set o servers S = {,..., s n }. Then, there also exists an assignment or the set o servers S = {,..., s j, s, s j+,..., s n }, namely a i,k, k < j a i,k = a i,j + a i,j+, k = j a i,k+, k > j By the inuction hypothesis, Algorithm ins such an assignment. However, we have assume that Algorithm oes not compute an assignment or m+ storage evices. Then, we have prove that or the given iles the algorithm behaves ientical as in the case o m storage evices. So, it shoul also not compute an assignment or m storage evices which is a contraiction. So, it always ins a solution i it exists.

6 IV. CONCLUSION We have presente a solution or optimally istributing iles in a istribute storage system or meia streaming. Our Algorithm optimally istributes iles on in O(nm + m log m), thus is a vast improvement over an o ar programming solution. This allows storing iles with known banwith requirement to a istribute storage system such that reistributing ata ater the assignment is never necessary. REFERENCES [] Giuseppe DeCania, Deniz Hastorun, Maan Jampani, Gunavarhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, an Werner Vogels. Dynamo: Amazon s highly available key-value store. SIGOPS Operating Systems Review, (6):5, 7. [] Peter Druschel an Antony I. T. Rowstron. Past: A large-scale, persistent peer-to-peer storage utility. In HotOS, pages 75 8,. [] Sanjay Ghemawat, Howar Gobio, an Shun-Tak Leung. The google ile system. In SOSP : Proceeings o the Nineteenth ACM Symposium on Operating Systems Principles, page9, New York, NY, USA,. ACM. [] Davi Karger, Eric Lehman, Tom Leighton, Rina Panigrahy, Matthew Levine, an Daniel Lewin. Consistent hashing an ranom trees: istribute caching protocols or relieving hot spots on the worl wie web. In STOC 97: Proceeings o the twenty-ninth annual ACM symposium on Theory o computing, pages 65 66, New York, NY, USA, 997. ACM. [5] John Kubiatowicz, Davi Binel, Yan Chen, Steven Czerwinski, Patrick Eaton, Dennis Geels, Ramakrishna Gummai, Sean Rhea, Hakim Weatherspoon, Chris Wells, an Ben Zhao. Oceanstore: An architecture or global-scale persistent storage. In ASPLOS-IX: Proceeings o the Ninth International Conerence on Architectural Support or Programming Languages an Operating Systems, page9, New York, NY, USA,. ACM. [6] Tobias Langner, Christian Schinelhauer, an Alexaner Souza. Optimal ile-istribution in heterogeneous an asymmetric storage networks. In Proceeings o the 7th international conerence on Current trens in theory an practice o computer science, SOFSEM, page68 8, Berlin, Heielberg,. Springer-Verlag. [7] Davi A. Patterson, Garth Gibson, an Rany H. Katz. A case or reunant arrays o inexpensive isks (rai). In SIGMOD 88: Proceeings o the 988 ACM SIGMOD International Conerence on Management o Data, page9 6, New York, NY, USA, 988. ACM. [8] Christian Schinelhauer an Gunnar Schomaker. Weighte istribute hash tables. In SPAA 5: Proceeings o the 7th ACM Symposium on Parallelism in Algorithms an Architectures, page8 7, Las Vegas, Nevaa, USA, 7 - July 5. ACM Press, New York, NY, USA. [9] Christian Schinelhauer an Gunnar Schomaker. SAN optimal multi parameter access scheme. In ICNICONSMCL 6: Proceeings o the International Conerence on Networking, International Conerence on Systems an International Conerence on Mobile Communications an Learning Technologies, page 8, Washington, DC, USA, 6. IEEE Computer Society. [] Steen Schott. Datenverteilung au heterogene speicher unter em atenstrom-ansatz. Master s thesis, Albert-Luwigs-Universität Freiburg, Germany,. [] Xiaohui Shen an Alok Chouhary. DPFS: A istribute parallel ile system. In ICPP : Proceeings o the International Conerence on Parallel Processing, page 5, Washington, DC, USA,. IEEE Computer Society. [] The Wuala Project. Wuala [On; accesse July ]. b b b b b b b b b b banwith s banwith s banwith s banwith s s stops ile ile ile ile ile Fig.. Sweep Line On Algorithm istributing ile onto servers,..., s.