Optimizing Content Retieval Delay fo LT-based Distibuted Cloud Stoage Systems Haifeng Lu, Chuan Heng Foh, Yonggang Wen, and Jianfei Cai School of Compute Engineeing, Nanyang Technological Univesity, Singapoe 639798 Email: {lu0007ng, aschfoh, ygwen, asjfcai}@ntu.edu.sg Abstact Among diffeent setups of cloud stoage systems, fountain-codes based distibuted cloud stoage system povides eliable online stoage solution though placing coded content fagments into multiple stoage nodes. Luby Tansfom (LT) code is one of the popula fountain codes fo stoage systems due to its efficient ecovey. Howeve, to ensue high success decoding of fountain codes based stoage, etieval of additional fagments is equied, and this equiement intoduces additional delay, which is citical fo content etieval o downloading applications. In this pape, we show that multiple-stage etieval of fagments is effective to educe the content-etieval delay. We fist develop a delay model fo vaious multiple-stage etieval schemes applicable to ou consideed system. With the developed model, we study optimal etieval schemes given the success decodability equiement. Ou numeical esults demonstate that the content-etieval delay can be significantly educed by optimally scheduling packet equests in a multi-stage fashion. I. INTRODUCTION Cloud stoage systems povide a scalable online stoage solution to end uses who equie flexible amount of stoage space but do not wish to own and maintain stoage infastuctue [], []. Cloud stoage systems consist of a collection of stoage nodes that ae connected though pivate o public Intenet. A content such as a lage file o a video is often fagmented and distibuted in a set of stoage nodes. To offe high eliability and availability of stoage sevices, edundancy of contents may be employed. Fagments of contents may be simply eplicated and stoed in diffeent stoage nodes to achieve edundancy. Easue codes may be used fo stoage edundancy to impove stoage efficiency of the system. In a distibuted stoage system with easue coding, a lage content is fist divided into a numbe of fagments, say k fagments, usually of same size. These fagments undego an encoding pocedue specified by a paticula easue coding to poduce an enlaged numbe of coded fagments, say n coded fagments, whee n k is the stoage edundancy. These coded fagments ae distibuted into diffeent stoage nodes fo impoved fault toleance. In an optimal pefoming easue code, the oiginal content can be etieved by obtaining any of the k coded fagments to econstuct the content. In geneal, thee ae two eseach effots in designing an efficient distibuted stoage system with easue codes fo obustness. The fist eseach effot falls on the design of enhanced easue codes that impove system pefomance. In [3], Hafne pesented Weave codes as a highly fault toleant easue code fo stoage systems. By optimizing the Cauchy distibution matix, Plank et al. intoduced an enhanced Reed- Solomon codes fo netwoked stoage system [4]. In [5], Huang et al. poposed Pyamid codes whee they exploed employment of nested easue codes as a single code. Oggie poposed Self-Repaiing codes et al. [6] that achieves local decodability suitable fo epaiing of fagments in distibuted stoage systems. Anothe eseach effot focuses on the design of stoage system opeational optimization based on existing easue codes to achieve some specific optimal pefomance matices. These opeational designs may include stoage edundancy ovehead optimization, stoage allocation and epai, optimal etieval of stoage, and othes. In [7], Sadai et al. investigated the optimal stoage allocation with limited stoage budget consideing Maximum Distance Sepaable (MDS) codes. Leong et al. studied stoage allocation fo high eliability [8]. Dimakis et al. intoduced Regeneating codes fo epaiing of missing fagments consideing linea netwok coding [9]. The fist study investigating the elationship between stoage system setup and etieval delay is epoted in [0] by Leong et al. whee the authos studied elationship between stoage allocation stategy and etieval delay. Ou eseach wok falls unde this effot of optimizing etieval delay fo a distibuted stoage system. As high pefomance easue codes often incu high complexity of computational, low computational complexity easue codes appea advantageous in pactical usage. Among vaious classes of easue codes, fountain codes ae a class that offes flexible edundancy due to its ateless popety and elatively low computational complexity. One popula ealization of fountain codes is Luby Tansfom (LT) codes []. LT codes enjoys elatively low computational complexity of O(k ln(k/δ)). Howeve, to achieve this elatively low computational complexity, additional encoded fagments in the ode of O( k ln (k/δ)) ae needed fo LT decode to achieve a successful decoding pobability of δ. The need fo additional encoded fagments adds stess to the bandwidth and intoduces delay in the etieval. In paticula, to ensue a high pobability of successful decoding pobability, a stoage collecto opeating LT decode equies additional encoded fagments fom the stoage system. Howeve, these additional encoded fagments add delay in stoage etieval, which is citical fo content etieval o downloading applications. Thus a tadeoff between the successful decoding pobability and stoage etieval delay
Fig.. Aival pocess of LT encoded packets seen at the potal. Fig.. System Achitectue fo a distibuted data stoage system. exists. In this pape, we study this tadeoff and intoduce a multiple stage stoage etieval scheme whee we show that stoage etieval delay can be educed without sacificing the pefomance on decoding pobability. We futhe demonstate the optimal etieval setup fo the case of a -stage etieval. With this optimal etieval scheme, we believe it will benefit pactical cloud stoage systems though poviding bette QoS in tems of etieval delay. The emainde of this pape is oganized as follows. In Section II, we intoduce the system achitectue and pesent the fomal definition of the optimal etieval scheme poblem. Section III pesents a igoous analysis on the delay fo diffeent etieval scheme togethe with simulation validation. We then demonstate the tadeoff of the delay and pobability of successful decoding in Section IV. Finally, some impotant conclusions ae dawn in Section V. II. SYSTEM ARCHITECTURE AND PROBLEM STATEMENT In this section, we fist pesent a detailed desciption of a distibuted data stoage system, in which LT-encoded data fagments ae spead out acoss a pool of stoage nodes. Following that, we fomulate an optimal file etieval poblem, which aims to minimize the etieval delay by stategically scheduling packet etieval equests. A. System Achitectue We focus on a distibuted data stoage system (e.g., an community-based cloud stoage system as in Tahoe-LAFS []) in which each stoage node stoes both LT fagments o packets and nomal packets fo diffeent contents. As illustated in Fig., the uses can stoe and etieve a file fom the cloud stoage system, via a dedicated potal. When stoing a file into the cloud, the potal encodes the file with the LT code and speads all the encoded fagments acoss diffeent stoage nodes fo eliability. Upon eceiving the file etieval equest, the potal sends a few packet etieval equests to diffeent stoage nodes, and eassemble the list of eceived packets into the file and fowad it to the use. Hee we use the tem packet intechangeably with fagment as we assume that an IP packet caies an LT encoded fagment in the system. We assume that thee is only one use who will equest LT packets. The taffics geneated by the equests fom the est uses ae teated as ambient taffics. Fig. shows a snapshot of the consideed system. B. Taffic Model In the potal, we conside a packet tansmission queue with a fixed pocessing ate of. The ambient taffic aives with an exponential inte-packet time (i.e., an aival ate of λ). The length of packets associated with the ambient taffic is L, which is a andom vaiable with mean l and vaiance σ. Moeove, stoage nodes tansmit LT-encoded packets back to the potal, with an exponential inte-packet aival time (o ate of θ). The length of LT encoded packets is a constant denoted by l LT. To ecove a file of k oiginal packets, the use may equest n encoded packets fom the potal in ode to achieve successful decoding pobability p. Afte the potal eceives the equest fo n LT encoded packets, it etieves these packets fom the stoage nodes. The equested n packets will then aive at the potal at a andom time which follows exponential distibution with a mean of /θ. Accoding to [3], the aival ate of i-th LT packets is (n i +)θ. This aival pocess of LT packets is illustated in Fig.. C. Poblem Statement In the consideed distibuted stoage system, the bottleneck lies on the potal, because a lage numbe of uses can issue thei file equests to the potal. In this pape, we focus on the file etieval delay, defined as the duation between the time fo potal eceiving a LT-encoded file equest and the time fo the last LT packet leaves the potal. We aim to minimize the file etieval delay by stategically scheduling the LT packet equests. The optimal file etieval poblem is stated as follows. We assume a pobabilistic file etieval model. Specifically, in ode to achieve the pobability of successful decoding of p, the potal needs to equest n LT encoded packets. Taditionally, the n packets ae equested in one shot and the potal waits fo all these packets to decode and etieve the file. This scheme is efeed to as an one-stage equest scheme. In this pape, we popose to use a multiple stage equest scheme to impove the file etieval pefomance. Specifically, the potal can divide the equest into t stages, each consisting of equests fo n,n,,n t packets ( i n i = n). If the potal successfully decodes the file at stage m, it will stop equesting the est t i=m+ n i packets. Intuitively, multiple stages of packet etieval incus additional delay in stoage etieval. Inteestingly, we found that
3 (a) One stage case (b) Two stage case Fig. 3. Components of delay..4..4..4. Simualtion 50 60 70 80 90 00 No. of equested packets (a) λ =0,θ =5 50 60 70 80 90 00 No. of equested packets 50 60 70 80 90 00 No. of equested packets Fig. 4. Delay in one stage equest. multiple stages of packet etieval can outpefom a single etieval in tems of delay. This is because in multiple stages of packet etieval, etieving a just adequate numbe of fagments may aleady give sufficiently high success decodability which eliminates the need fo futhe ounds of etieval. In case that the decoding fails, a futhe ound of etieval may take place. By popely configue the multiple stage packet etieval scheme, the gain in delay with a multiple stage etieval outuns the loss due to additional ounds of etieval. We demonstate this by pesenting an optimization of two-stage equest scheme. III. FILE-RETRIEVAL DELAY ANALYSIS We chaacteize the aveage file-etieval delay fo diffeent equest schemes in the pesence of ambient taffic. We use compute simulation to validate ou analytical esults. In simulations, we set that the length of packets in the ambient taffic follows an exponential distibution. Although it is a simplified model, the insights obtained with this simple model can be applied to guide pactical system design. Table I lists the default values of some paametes in the simulato. All the pesented simulation esults in the following sections ae aveaged fom 000 tails. TABLE I DEFAULT VALUES IN SIMULATION Notation l l LT Value 04 bits 04 bits 00 kbps A. Delay Analysis fo One Stage Request Scheme We fist investigate the file-etieval delay D (n) fo n LT packets in the one stage equest case. This delay consists of two pats as shown in Fig. 3(a). The fist pat aises fom the taffic aiving befoe the LT equest and the second pat aises fom the taffic aiving afte the LT equest. The fist pat only contains ambient taffic. The fist pat is teated as a classical M/G/ queue with delay denoted by W. The second pat can be futhe divided into two sub-pats. Fistly, it contains a constant which is the tansmission time of n LT packets, which can be deived as nl LT. The est is the time to pocess the ambient taffics which aive at the potal duing the inteaival time of each LT packet. This pat is denoted by T. Thus, the delay D (n) can be expessed as D (n) =W + nl LT + T. () Taking an expectation on both sides, we obtain E(D (n)) = E(W )+E(T )+ nl LT = λ(σ + l ) ( λ l ) + λl n E(ζi+ ζi )+ nl LT i=0 = Γ+ λl n iθ + nl LT i= Γ+ λl θ (ln(n) + ) + nl LT, () whee Γ is a constant, ζi is aiving time fo the ith LT encoded packet and ζ0 =0. In Fig. 4, we plot the numeical file etieval delay, compaed with the simulation esults, as a function of the numbe of packets equest, fo vaious taffic loads. Notice that the numeical esults match the simulation esults well, veifying the applicability of ou deived file-delay in (). The esults also demonstate that the file-etieval delay gows linealy with the numbe of packets equested.
4 B. Delay Analysis fo Multiple Stage Request In this subsection, we fist investigate the file-etieval delay fo the two stage equest scheme. The packet aival pocess is illustated in Fig. 3(b). Suppose in the fist stage, the use equests n LT encoded packets. The delay fo the fist stage is D (n ). Afte decoding these n packets, if the use fails to decode the oiginal file, it continues to equest the n packets. Hee we assume that duing the decoding pocess of n packets, the tansmission queue in the potal has aleady etuned to the steady state. This assumption is easonable since it does take some time fo the use to detemine if the n LT-encoded packets ae decodable. Thus, the delay fo the second stage is identical to the fist stage except fo the numbe of encoded packets the use equests. As a esult, the oveall file-etieval delay fo the two stage equest case is given by D (n,n ) = f(n )D (n ) + ( f(n ))(D (n )+D (n )), (3) whee f(n) is the pobability of successful decoding with n LT encoded packets. The detemination of f(n) is given sepaately in [4] and [5]. Hee we adopt the model intoduced in [5] due to its computational efficiency. In Fig. 5, we plot the successful decoding pobability as a function of the numbe of eceived packets when k =50and k = 00 espectively. 0. 0 50 60 70 80 90 00 No. of eceived packets (a) k =50 0. 0 00 0 40 60 80 00 No. of eceived packets (b) k = 00 Fig. 5. The pobability of successful decoding as a function of the numbe of eceived LT-encoded packets. Using the esults fom Fig. 5, we can obtain the numbe of equied packets fo a tageted decoding pobability. We plot in Figs. 6-7 the file-etieval delay fo the two-stage equest scheme, as a function of the numbe of packets equested in the fist stage, unde diffeent setting of k, n and taffic loads. Notice that the numeical esults match the simulation esults well, egadless of the taffic load. Moeove, in all esults, the file-etieval delay fist deceases and then inceases as the numbe of packets equested in the fist stage inceases. This consistent tend indicates a potential fo file-etieval delay minimization by choosing an appopiate numbe of packets equested in the fist stage. We shall addess this optimization in Section IV-A. The esult fo the two stage equest scheme can be genealized into abitay t stage equest. Specifically, the delay expession fo t stage equest can be defined ecusively, D t (n,n,,n t ) = ( f(n ))(D (n ) (4) + D t (n,,n t )) + f(n )D (n ), whee the initial values fo D and D ae defined in () and (3). IV. OPTIMAL SCHEDULING FOR PACKET RETRIEVAL In this section, we will fist investigate the optimal scheduling of packet equests to minimize the file-etieval delay fo two-stage equest scheme, and then pesent a fundamental tade-off between the file etieval delay and the pobability of file decodability. A. Optimal Request Scheme The esults in Fig. 6-7 suggest an optimal two stage equest scheme fom (3). Solving the optimization poblem yields the optimal numbe of packets equests in the fist stage fo a twostage equest scheme. In Fig. 8-9, we plot, fo the optimal twostage equest scheme, the atio (n /n) between the numbe of equested packets in the fist stage and the total numbe of packets needed, as a function of the decoding pobability, fo diffeent taffic loads. Fom these figues, we make an obsevation. We see that the optimal potion of packets equested in the fist stage deceases as the taget pobability of successful decoding inceases. This obsevation can be undestood as follows. Fom the aspect of decodability pobability, evey packet contibutes in the decoding pocess. Fom Fig. 5, we see an unequal contibution of each packet to the decodability pobability. The contibution of an additional packet to the decodability pobability is high when the pobability stays at a lowe level. As the pobability pogesses highe, the contibution of each additional packet dops. Notice fom Fig. 4 that delay in one stage equest scheme inceases linealy with the numbe of equested packets. Instead of equesting all the packets equied by the taget decodability pobability at once, equesting a small potion of the numbe of packets needed in the fist stage will geneate a decoding pobability that contibutes most to the final decoding pobability; while the maginal contibution fom the second stage is elatively small. As a esult, a twostage equest scheme helps keep the aveage delay lowe than the one-stage equest scheme. B. Delay-Decodability Tadeoff We now investigate the fundamental tadeoff between the tageted decoding pobability and the file-etieval delay fo diffeent equest schemes. This delay-decodability tadeoff is impotant since it helps uses to make ational equest scheme depending on the equiement of thei applications. Using the esults fom pevious two subsections, we plot the afoementioned tade-off fo the one-stage equest scheme and the two-stage equest schemes, in Fig. 0-, fo diffeent system loads.
5.05.05.05 5 5 5 5 50 60 70 80 90 00 No. of equest packets in fist stage (a) λ =0,θ =5 5 50 60 70 80 90 00 No. of equested packets in fist stage 5 50 60 70 80 90 00 No. of equested packets in fist stage Fig. 6. Delay in two stage equest when k =50and n = 00.....8.6.8.6.8.6.4.4.4 00 0 40 60 80 00 No. of equested packets in fist stage 00 0 40 60 80 00 No. of equested packets in fist stage 00 0 40 60 80 00 No. of equested packets in fist stage (a) λ =0,θ =5 Fig. 7. Delay in two stage equest when k = 00 and n = 00. 5 5 5 5 5 5 5 5 5 0.5 (a) λ =0,θ =5 0.5 0.5 Fig. 8. of numbe of equested LT packets in fist stage when k =50. As obseved in these figues, the delay fo one-stage equest inceases shaply when the decoding pobability cosses 90%. In compaison, fo the two-stage equest scheme, the delay is consistently lowe. This advantage eflects the gain fom being able to decode in the fist stage with an adequate instead of excessive numbe of LT packets. Based on the epoted delay-decodability tadeoff, use can choose an appopiate taget of decodability pobability, i.e. fo some delay sensitive applications like steam video, a lowe decodability pobability can be adopted such that the aveage etieval delay can be maintained unde a cetain theshold. V. CONCLUSION In this pape, we investigate the poblem of optimal file etieval unde a distibuted cloud stoage system. The file is fist LT-encoded and spead out into a list of distibuted stoage nodes. When etieval, the potal schedules the packet equest in a multi-stage manne, with an objective to minimize the aveage file etieval delay. We developed an accuate model to chaacteize the aveage file-etieval delay fo diffeent equest stategies. Using this model, we deived an optimal two stage equest scheme fo a given decoding pobability. Both simulation and numeical esult confim that this optimal scheme can educe the aveage delay damatically. We believe such delay-optimized etieval scheme can benefit pactical distibuted cloud stoage systems though poviding bette QoS in tems of etieval delay. REFERENCES [] Amazon S3. http://aws.amazon.com/s3/.
6 0.5 0.5 (a) λ =0,θ =5 0.5 0.5 0.5 0.5 Fig. 9. of numbe of equested LT packets in fist stage when k = 00..6.4.6.4.6.4. one stage equest two stage equest. one stage equest two stage equest. one stage equest two stage equest 0.5 (a) λ =0,θ =5 0.5 0.5 Fig. 0. Tadeoff between decoding pobability and delay when k =50..5.5 two stage equest one stage equest.5.5 two stage equest one stage equest.5.5 two stage equest one stage equest 0.5 (a) λ =0,θ =5 0.5 0.5 Fig.. Tadeoff between decoding pobability and delay when k = 00. [] EMC Atmos Online Stoage. http://www.atmosonline.com/. [3] J. Hafne, Weave codes: highly fault toleant easue codes fo stoage systems, in Poc. the 4th confeence on USENIX Confeence on File and Stoage Technologies. USENIX Association, 005, pp. 6 6. [4] J. Plank and L. Xu, Optimizing cauchy Reed-Solomon codes fo faulttoleant netwok stoage applications, in IEEE Int. Symp. Netwok Computing and Applications. IEEE, 006, pp. 73 80. [5] J. L. Cheng Huang, Minghua Chen, Pyamid codes: Flexible schemes to tade space fo access efficiency in eliable data stoage systems, in IEEE Int. Symp. Netwok Computing and Applications. IEEE, 007, pp. 79 86. [6] F. Oggie and A. Datta, Self-epaiing homomophic codes fo distibuted stoage systems, in Poc. IEEE INFOCOM. IEEE, 00, pp. 5 3. [7] M. Sadai, R. Restepo, F. Feki, and E. Soljanin, Memoy allocation in distibuted stoage netwoks, in Poc. Int. Symp. Infomation Theoy ISIT. IEEE, 00, pp. 958 96. [8] D. Leong, A. Dimakis, and T. Ho, Distibuted stoage allocation fo high eliability, in IEEE Int. Confeence on Communications (ICC). IEEE, 00, pp. 6. [9] A. Dimakis, P. Godfey, Y. Wu, M. Wainwight, and K. Ramchandan, Netwok coding fo distibuted stoage systems, Infomation Theoy, IEEE Tansactions on, vol. 56, no. 9, pp. 4539 455, 00. [0] D. Leong, A. Dimakis, and T. Ho, Distibuted stoage allocations fo optimal delay, in Poc. Int. Symp. Infomation Theoy ISIT, 0. [] M. Luby, LT codes, in Poc. 43d Annual IEEE Symp. Foundations of Compute Science, 00, pp. 7 80. [] Z. Wilcox-O Hean and B. Wane, Tahoe: the least-authoity filesystem, in Poceedings of the 4th ACM intenational wokshop on Stoage secuity and suvivability, 008, pp. 6. [3] A. Rényi, On the theoy of ode statistics, Acta Mathematica Hungaica, vol. 4, pp. 9 3, 953. [4] R. Kap, M. Luby, and A. Shokollahi, Finite length analysis of LT codes, in Poc. Int. Symp. Infomation Theoy ISIT, 004, p. 39. [5] E. Maneva and A. Shokollahi, New model fo igoous analysis of LTcodes, in Poc. Int. Symp. Infomation Theoy ISIT, 006, pp. 677 679.