# Rethinking Erasure Codes for Cloud File Systems: Minimizing I/O for Recovery and Degraded Reads

Save this PDF as:

Size: px
Start display at page:

## Transcription

4 Symbol / w words Symbol w bit words Sealed Block k data disks r symbols Figure 2: Relationship between words, symbols and sealed blocks. code, it does not map directly to a disk system. In reality, it makes sense for each symbol in a sealed block to be much larger in size, on the order of kilobytes or megabytes, and for each symbol to be partitioned into w- bit words, which are encoded and decoded in parallel. Figure 2 depicts such a partitioning, where each symbol is composed of multiple words. When w = 1, this partitioning is especially efficient, because machines support bit operations like exclusive-or (XOR) over 64-bit and even 128-bit words, which in effect perform 64 or 128 XOR operations on 1-bit words in parallel. When w = 1, the arithmetic is modulo 2: addition is XOR, and multiplication is AND. When w > 1, the arithmetic employed is Galois Field arithmetic, denoted GF (2 w ). In GF (2 w ), addition is still XOR; however multiplication is more complex, requiring a variety of implementation techniques that depend on hardware, memory, co-processing elements and w [17]. 3.1 Matrix-Vector Definition All erasure codes may be expressed in terms of a matrixvector product. An example is pictured in Figure 3. This continues the example from Figure 1, where k = 6, m = 3 and r = 4; In this picture, the erasure code is defined precisely. This is a Cauchy Reed-Solomon code [6] optimized by the Jerasure library [38]. The word size, w equals one, so all symbols are treated as bits and arithmetic is composed solely of the XOR operation. The kr symbols of data are organized as a kr-element bit vector. They are multiplied by a nr kr Generator matrix G T. 1 The product is a vector, called the codeword, with nr elements. These are all of the symbols in the stripe. Each collection of r symbols in the vector is stored on a different disk in the system. Since the the top kr rows of G T compose an identity matrix, the first kr symbols in the codeword contain the 1 The archetypical presentation of erasure codes [26, 29, 32] typically uses the transpose of this matrix; hence, we call this matrix G T. data. The remaining mr symbols are calculated from the data using the bottom mr rows of the Generator matrix. When up to m disks fail, the standard methodolgy for recovery is to select k surviving disks and create a decoding matrix B from the kr rows of the Generator matrix that correspond to them. The product of B 1 and the symbols in the k surviving disks yields the original data [6, 20, 33]. There are many MDS erasure codes that apply to storage systems. Reed-Solomon codes [40] are defined for all values of k and m. With a Reed-Solomon code, r = 1, and w must be such that 2 w n. Generator matrices are constructed from a Vandermonde matrix so that any k k subset of the Generator matrix is invertible. There is quite a bit of reference material on Reed-Solomon codes as they apply to storage systems [33, 36, 6, 41], plus numerous open-source Reed- Solomon coding libraries [42, 38, 30, 31]. Cauchy Reed-Solomon codes convert Reed-Solomon codes with r = 1 and w > 1 to a code where r = w and w = 1. In doing so, they remove the expensive multiplication of Galois Fields and replace it with additional XOR operations. There are an exponential number of ways to construct the Generator matrix of a Cauchy Reed-Solomon code. The Jerasure library attempts to construct a matrix with a minimal number of non-zero entries [38]. It is these matrices that we use in our examples with Cauchy Reed-Solomon codes. For m = 2, otherwise known as RAID-6, there has been quite a bit of research on constructing codes where w = 1 and the CPU performance is optimized. EVENODD [3], RDP [11] and Blaum-Roth [5] codes all require r + 1 to be a prime number such that k r + 1 (EVENODD) or k r. The Liberation codes [34] require r to be a prime number and k r, and the Liber8tion code [35] is defined for r = 8 and k r. The latter three codes (Blaum-Roth, Liberation and Liber8tion) belong to a family of codes called Minimum Density codes, whose Generator matrices have a provably minimum number of ones. Both EVENODD and RDP codes have been extrapolated to higher values of m [2, 4]. We call these Generalized EVENODD and RDP. With m = 3, the same restrictions on r apply. For larger values of m, there are additional restrictions on r. The STAR code [24] is an instance of the generalized EVENODD codefor m = 3, where recovery is performed without using the Generator matrix. All of the above codes have a convenient feature that disk C 0 is constructed as the parity of the data disks, as in RAID-4/5. Thus, the r rows of the Generator matrix immediately below the identity portion are composed of k (r r) identity matrices. To be consistent with these RAID systems, we will refer to disk C 0 as the P drive. 4

5 Figure 3: The matrix-vector representation of an erasure code. The parameters are the same as Figure 1: k = 6, m = 3 and r = 4. Symbols are one bit (i.e. w = 1). This is a Cauchy Reed-Solomon code for these parameters. 4 Optimal Recovery of XOR-Based Erasure codes When a data disk fails in an erasure coded disk array, it is natural to reconstruct it simply using the P drive. Each failed symbol is equal to the XOR of corresponding symbols on each of the other data disks, and the parity symbol on the P disk. We call this methodology Reading from the P drive. It requires k symbols to be read from disk for each decoded symbol. Although it is straightforward both in concept and implementation, in many cases, reading from the P drive requires more I/O than is necessary. In particular, depending on the erasure code, there are savings that can be exploited when multiple symbols are recovered in the same stripe. This effect was first demonstrated by Xiang et al. in RDP systems in which one may reconstruct all the failed blocks in a stripe by reading 25 percent fewer symbols than reading from the P drive [51]. In this section, we approach the problem in general. 4.1 Algorithm to Determine the Minimum Number of Symbols for Recovery We present an algorithm for recovering from a single disk failure in any XOR-based erasure code with a minimum number of symbols. The algorithm takes as input a Generator matrix whose symbols are single bits and the identity of a failed disk and outputs equations to decode each failed symbol. The inputs to the equations are the symbols that must be read from disk. The number of inputs is minimized. The algorithm is computationally expensive for the systems evaluated for this paper, each instantiation took from seconds to hours of compute-time. However, for any realistic storage system, the number of recovery scenarios is limited, so that the algorithm may be run ahead of time, and the results may be stored for when they are required by the system. We explain the algorithm by using the erasure code of Figure 4 as an example. This small code, with k = m = r = 2, is not an MDS code; however its simplicity facilitates our explanation. We label the rows of G T as R i, 0 i < nr. Each row R i corresponds to a data or coding symbol, and to simplify our presentation, we will refer to symbols using R i rather than d i,j or c i,j. Consider a set of symbols in the codeword whose corresponding rows in the Generator matrix sum to a vector of zeroes. One example is {R 0, R 2, R 4 }. We call such a set of symbols a decoding equation, because the fact their rows sum to zero allows us to decode any one symbol in the set as long as the remaining symbols are not lost. Suppose that we enumerate all decoding equations for a given Generator matrix, and suppose that some subset F of the codeword symbols are lost. For each symbol R i F, we can determine the set E i of decoding equations for R i. Formally, an equation e i E i if e i F = {R i }. For example, the equation represented by the set {R 0, R 2, R 4 } may be a decoding equation in e 2 so long as neither R 0 nor R 4 is in F. Figure 4: An example erasure code to explain the algorithm to minimize the number of symbols required to recover from failures. 5

6 We can recover all the symbols in F by selecting one decoding equation e i from each set E i, reading the nonfailed symbols in e i and then XOR-ing them to produce the failed symbol. To minimize the number of symbols read, our goal is to select one equation e i from each E i such that the number of symbols in the union of all e i is minimized. For example, suppose that a disk fails, and both R 0 and R 1 are lost. A standard way to decode the failed bits is to read from the P drive and use coding symbols R 4 and R 5. In equation form, F = {R 0, R 1 } e 0 = {R 0, R 2, R 4 } and e 1 = {R 1, R 3, R 5 }. Since e 0 and e 1 have distinct symbols, their union is composed of six symbols, which means that four must be read for recovery. However, if we instead use {R 1, R 2, R 7 } for e 1, then (e 0 e 1 ) has five symbols, meaning that only three are required for recovery. Thus, our problem is as follows: Given F sets of decoding equations E 0, E 1,... E F 1, we wish to select one equation from each set such that the size of the union of these equations is minimized. Unfortunately, this problem is NP-Hard in F and E i. 2 However, we can solve the problem for practical values of F and E i (typically less than 8 and 25 respectively) by converting the equations into a directed, weighted graph and finding the shortest path through the graph. Given an instance of the problem, we convert it to a graph as follows. First, we represent each decoding equation in set form as an nrelement bit string. For example, {R 0, R 2, R 4 } is represented by Each node in the graph is also represented by an nrelement bit string. There is a starting node Z whose string is all zeroes. The remaining nodes are partitioned into F sets, labeled S 0, S 1,... S F 1. For each equation e 0 E 0, there is a node s 0 S 0 whose bit string equals e 0 s bit string. There is an edge from Z to each s 0 whose weight is equal to the number of ones in s 0 s bit string. For each node s i S i, there is an edge that corresponds to each e i+1 E i+1. This edge is to a node s i+1 S i+1 whose bit string is equal to the bitwise OR of the bit strings of s i and e i+1. The OR calculates the union of the equations leading up to s i and e i+1. The weight of the edge is equal to the difference between the number of ones in the bit strings of s i and s i+1. The shortest path from Z to any node in S F 1 denotes the minimum number of elements required for recovery. If we annotate each edge with the decoding equation that creates it, then the shortest path contains the equations that are used for recovery. To illustrate, suppose again that F = {R 0, R 1 }, meaning f 0 = R 0 and f 1 = R 1. The decoding equations 2 Reduction from Vertex Cover. for E 0 and E 1 are denoted by e i,j where i is the index of the lost symbol in the set F and j is an index into the set E i. E 0 and E 1 are enumerated below: E 0 E 1 e 0,0 = e 1,0 = e 0,1 = e 1,1 = e 0,2 = e 1,2 = e 0,3 = e 1,3 = These equations may be converted to the graph depicted in Figure 5, which has two shortest paths of length five: {e 0,0, e 1,2 } and {e 0,1, e 1,0 }. Both require three symbols for recovery: {R 2, R 4, R 7 } and {R 3, R 5, R 6 }. While the graph clearly contains an exponential number of nodes, one may program Dijkstra s algorithm to determine the shortest path and prune the graph drastically. For example, in Figure 5, the shortest path will be discovered before the the dotted edges and grayed nodes are considered by the algorithm. Therefore, they may be pruned. Figure 5: The graph that results when R 0 and R 1 are lost. 4.2 Algorithm for Reconstruction When data disk i fails, the algorithm is applied for F = {d i,0,..., d i,r 1 }. When coding disk j fails, F = {c j,0,..., c j,r 1 }. If a storage system rotates the identities of the disks on a stripe-by-stripe basis, then the average number of symbols for all failed disks multiplied by the total number of stripes gives a measure of the symbols required to reconstruct a failed disk. 4.3 Algorithm for Degraded Reads To take maximum advantage of parallel I/O, we assume that contiguous symbols in the file system are stored on 6

8 Figure 9: Reconstructing disk 0 when it fails, using Rotated Reed-Solomon coding for k = 6, m = 3, r = 4. Figure 7: A Rotated Reed-Solomon code for k = 6, m = 2 and r = 3. the P drive, we need to read d 0,0 through d 4,0 plus c 0,0 to reconstruct d 5,0. Then we read the non-failed symbols d 0,1, d 1,1 and d 2,1. The penalty is 5 symbols. With Rotated Reed-Solomon coding, d 5,0, d 0,1, d 1,1 and d 2,1 all participate in the equation for c 1,0. Therefore, by reading c 1,0, d 0,1, d 1,1, d 2,1, d 3,0 and d 4,0, one both decodes d 5,0 and reads the symbols that were required to be read. The penalty is only two symbols. Figure 8: A Rotated Reed-Solomon code for k = 6, m = 3 and r = 3. With whole disk reconstruction, when r is an even number, one can reconstruct any failed data disk by reading r 2 (k + k m ) symbols. The process is exemplified for k = 6, m = 3 and r = 4 in Figure 9. The first data disk has failed, and the symbols required to reconstruct each of the failed symbols is darkened and annotated with the equation that is used for reconstruction. Each pair of reconstructed symbols in this example shares four data symbols for reconstruction. Thus, the whole reconstruction process requires a total of 16 symbols, as opposed to 24 when reading from the P Drive. The process is similar for the other data drives. Reconstructing failed coding drives, however does not have the same benefits. We are unaware at present of how to reconstruct a coding drive with fewer than the maximum kr symbols. As an aside, when more than one disk fails, Rotated Reed-Solomon codes may require much more computation to recover than other codes, due to the use of matrix inversion for recovery. We view this property as less important, since multiple disk failures are rare occurrences in practical storage systems, and computational overhead is less important than the I/O impact of recovery. 5.1 MDS Constraints The Rotated Reed-Solomon code specified above in Section 5 is not MDS in general. In other words, there are settings of k, m, w and r which cannot tolerate the failure of any m disks. Below, we detail ways to constrain these variables so that the Rotated Reed-Solomon code is MDS. Each of these settings has been verified by testing all combinations of m failures to make sure that they may be tolerated. They cover a wide variety of system sizes, certainly much larger than those in use today. The constraints are as follows: m {2, 3} k 36, and k + m 2 w + 1 w {4, 8, 16} r {2, 4, 8, 16, 32} Moreover, when w = 16, r may be any value less than or equal to 48, except 15, 30 and 45. It is a matter of future research to derive general-purpose MDS constructions of Rotated Reed-Solomon codes. 6 Analysis of Reconstruction We evaluate the minimum number of symbols required to recover a failed disk in erasure coding systems with a variety of erasure codes. We restrict our attention to MDS codes, and systems with six data disks and either two or 8

9 Percentage of using P Disk Data Disks Only All Disks m=2 m=3 Density: Fraction of ones m=2 m=3 Cauchy RS (r=8) Cauchy RS (r=7) Cauchy RS (r=6) Cauchy RS (r=5) STAR (r=6) Rotated RS (r=6) Cauchy RS (r=4) Gen. RDP (r=6) Cauchy RS (r=3) Cauchy RS (r=4) Cauchy RS (r=5) Rotated RS (r=6) EVENODD (r=6) Cauchy RS (r=8) Cauchy RS (r=6) Cauchy RS (r=7) Liberation (r=7) Blaum-Roth (r=6) RDP (r=6) Liber8tion (r=8) Cauchy RS (r=8) Cauchy RS (r=7) Cauchy RS (r=6) Cauchy RS (r=5) STAR (r=6) Rotated RS (r=6) Cauchy RS (r=4) Gen. RDP (r=6) Cauchy RS (r=3) Cauchy RS (r=4) Cauchy RS (r=5) Rotated RS (r=6) EVENODD (r=6) Cauchy RS (r=8) Cauchy RS (r=6) Cauchy RS (r=7) Liberation (r=7) Blaum-Roth (r=6) RDP (r=6) Liber8tion (r=8) Figure 10: The minimum number of symbols required to reconstruct a failed disk in a storage system when k = 6 and m {2, 3}. three coding disks. We summarize the erasure codes that we test in Table 1. For each code, if r has restrictions based on k and m, we denote it in the table and include the actual values tested in the last column. All codes, with the exception of Rotated Reed-Solomon codes, are XOR codes, and all without exception define the P drive identically. Since there are a variety of Cauchy Reed- Solomon codes that can be generated for any value of k, m and r, we use the codes generated by the Jerasure coding library, which attempts to minimize the number of non-zero bits in the Generator matrix [38]. Code m Restrictions on r r tested EVENODD [3] 2 r + 1 prime k 6 RDP [11] 2 r + 1 prime > k 6 Blaum-Roth [5] 2 r + 1 prime > k 6 Liberation [34] 2 r prime k 7 Liber8tion [35] 2 r = 8, r k 8 STAR [24] 3 r + 1 prime k 6 Generalized RDP [2] 3 r + 1 prime > k 6 Cauchy RS [6] 2,3 2 r n 3-8 Rotated 2,3 None 6 Table 1: The erasure codes and values of r tested. For each code listed in Table 1, we ran the algorithm from section 4.1 to determine the minimum number of symbols required to reconstruct each of the k + m failed disks in one stripe. The average number is plotted in Figure 10. The Y-axis of these graphs are expressed as a percentage of kr, which represents the number of symbols required to reconstruct from the P drive. This is also the number of symbols required when standard Reed-Solomon coding is employed. In both sides of the figure, the codes are ordered from best to worst, and two bars are plotted: the average num- Figure 11: The density of the bottom mr rows of the Generator matrices for the codes in Figure 10. ber of symbols required when the failed disk is a data disk, and when the failed disk can be either data or coding. In all codes, the performance of decoding data disks is better than re-encoding coding disks. As mentioned in Section 5, Rotated Reed-Solomon codes require kr symbols to re-encode. In fact, the C 1 drive in all the RAID-6 codes require kr symbols to re-encode. Regardless, we believe that presenting the performance for data and coding disks is more pertinent, since disk identities are likely to be rotated from stripe to stripe, and therefore a disk failure will encompass all n decoding scenarios. For the RAID-6 systems, the minimum density codes (Blaum-Roth, Liberation and Liber8tion) as a whole exhibit excellent performance, especially when data disks fail. It is interesting that the Liber8tion code, whose construction was the result of a theory-less enumeration of matrices, exhibits the best performance. Faced with these results, we sought to determine if Generator matrix density has a direct impact on disk recovery. Figure 11 plots the density of the bottom mr rows of the Generator matrices for each of these codes. To a rough degree, density is correlated to recovery performance of the data disks; however the correlation is only approximate. The precise relationship between codes and their recovery performance is a direction of further research. Regardless, we do draw some important conclusions from the work. The most significant one is that reading from the P drive or using standard Reed-Solomon codes is not a good idea in cloud storage systems. If recovery performance is a dominant concern, then the Liber8tion code is the best for RAID-6, and Generalized RDP is the best for three fault-tolerant systems. 9

13 References [1] H. P. Anvin. The mathematics of RAID-6. people/hpa/raid6.pdf, [2] M. Blaum. A family of MDS array codes with minimal number of encoding operations. In IEEE International Symposium on Information Theory, September [3] M. Blaum, J. Brady, J. Bruck, and J. Menon. EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures. IEEE Transactions on Computing, 44(2): , February [4] M. Blaum, J. Bruck, and A. Vardy. MDS array codes with independent parity symbols. IEEE Transactions on Information Theory, 42(2): , February [5] M. Blaum and R. M. Roth. On lowest density MDS codes. IEEE Transactions on Information Theory, 45(1):46 59, January [6] J. Blomer, M. Kalfane, M. Karpinski, R. Karp, M. Luby, and D. Zuckerman. An XOR-based erasure-resilient coding scheme. Technical Report TR , International Computer Science Institute, August [7] V. Bohossian and J. Bruck. Shortening array codes and the perfect 1-Factorization conjecture. In IEEE International Symposium on Information Theory, pages , [8] D. Borthakur. The Hadoop distributed file system: Architecture and design. common/docs/current/hdfs-design.html, [9] E. Brewer. Lessons from giant-scale services. Internet Computing, 5(4), [10] B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, J. Haridas, C. Uddaraju, H. Khatri, A. Edwards, V. Bedekar, S. Mainali, R. Abbasi, A. Agarwal, M. Fahim ul Haq, M. Ikram ul Haq, D. Bhardwaj, S. Dayanand, A. Adusumilli, M. McNett, S. Sankaran, K. Manivannan, and L. Rigas. Windows Azure storage: A highly available cloud storage service with strong consistency. In Symposium on Operating Systems Principles, [11] P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar. Row diagonal parity for double disk failure correction. In Conference on File and Storage Technologies, March [12] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchandran. Network coding for distributed storage systems. IEEE Trans. Inf. Theor., 56(9): , September [13] A. L. Drapeau et al. RAID-II: A high-bandwidth network file server. In International Symposium on Computer Architecture, [14] B. Fan, W Tanisiriroj, L. Xiao, and G. Gibson. DiskReduce: RAID for data-intensive scalable computing. In Parallel Data Storage Workshop, [15] D. Ford, F. Labelle, F..I. Popovici, M. Stokely, V.-A. Truong, L. Barroso, C. Grimes, and S. Quinlan. Availability in globally distributed file systems. In Operating Systems Design and Implementation, [16] S. Ghemawat, H. Gobioff, and S. Leung. The Google file system. In ACM SOSP, [17] K. Greenan, E. Miller, and T. J. Schwartz. Optimizing Galois Field arithmetic for diverse processor architectures and applications. In Modeling, Analysis and Simulation of Computer and Telecommunication Systems, September [18] K. M. Greenan, X. Li, and J. J. Wylie. Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs. Mass Storage Systems and Technologies, [19] J. L. Hafner. Weaver codes: Highly fault tolerant erasure codes for storage systems. In Conference on File and Storage Technologies, [20] J. L. Hafner, V. Deenadhayalan, K. K. Rao, and J. A. Tomlin. Matrix methods for lost data reconstruction in erasure codes. In Conference on File and Storage Technologies, [21] M. Holland and G. A. Gibson. Parity declustering for continuous operation in redundant disk arrays. In Architectural Support for Programming Languages and Operating Systems. ACM, [22] R. Y. Hou, J. Menon, and Y. N. Patt. Balancing I/O response time and disk rebuild time in a RAID5 disk array. In Hawai i International Conference on System Sciences, [23] C. Huang, M. Chen, and J. Li. Pyramid codes: Flexible schemes to trade space for access efficiency in reliable data storage systems. Network Computing and Applications, [24] C. Huang and L. Xu. STAR: An efficient coding scheme for correcting triple storage node failures. IEEE Transactions on Computers, 57(7): , July [25] H. Jin, J. Zhang, and K. Hwang. A raid reconfiguration scheme for gracefully degraded operations. EuroMicro Conference on Parallel, Distributed, and Network-Based Processing, 0:66, [26] D. Kenchammana-Hosekote, D. He, and J. L. Hafner. REO: A generic RAID engine and optimizer. In Conference on File and Storage Technologies, pages , [27] O. Khan, R. Burns, J. S. Plank, and C. Huang. In search of I/O-optimal recovery from disk failures. In Workshop on Hot Topics in Storage Systems, [28] M. Luby, M. Mitzenmacher, A. Shokrollahi, D. Spielman, and V. Stemann. Practical loss-resilient codes. In 29th Annual ACM Symposium on Theory of Computing, pages , El Paso, TX, ACM. [29] F. J. MacWilliams and N. J. A. Sloane. The Theory of Error-Correcting Codes, Part I. North-Holland Publishing Company,

14 [30] Onion Networks. Java FEC Library v Open source code distribution: fec/javadoc/, [31] A. Partow. Schifra Reed-Solomon ECC Library. Open source code distribution: com/downloads.html, [32] W. W. Peterson and E. J. Weldon, Jr. Error-Correcting Codes, Second Edition. The MIT Press, [33] J. S. Plank. A tutorial on Reed-Solomon coding for faulttolerance in RAID-like systems. Software Practice & Experience, 27(9): , [34] J. S. Plank. The RAID-6 Liberation codes. In Conference on File and Storage Technologies, [35] J. S. Plank. The RAID-6 Liber8Tion code. Int. J. High Perform. Comput. Appl., 23: , August [36] J. S. Plank and Y. Ding. Note: Correction to the 1997 tutorial on Reed-Solomon coding. Software Practice & Experience, 35(2): , February [37] J. S. Plank, J. Luo, C. D. Schuman, L. Xu, and Z. Wilcox- OHearn. A performance evaluation and examination of open-source erasure coding libraries for storage. In Conference on File and Storage Technologies, [38] J. S. Plank, S. Simmerman, and C. D. Schuman. Jerasure: A library in C/C++ facilitating erasure coding for storage applications - Version 1.2. Technical Report CS , University of Tennessee, August [39] K. V. Rashmi, N. B. Shah, P. V. Kumar, and K. Ramachandran. Explicit construction of optimal exact regenerating codes for distributed storage. In Communication, Control, and Computing, [40] I. S. Reed and G. Solomon. Polynomial codes over certain finite fields. Journal of the Society for Industrial and Applied Mathematics, 8: , [41] L. Rizzo. Effective erasure codes for reliable computer communication protocols. ACM SIGCOMM Computer Communication Review, 27(2):24 36, [42] L. Rizzo. Erasure codes based on Vandermonde matrices. Gzipped tar file posted at rubrique.php3?id rubrique=10, [43] R. Rodrigues and B. Liskov. High availability in DHTS: Erasure coding vs. replication. In Workshop on Peer-to- Peer Systems, [44] B. Schroeder and G. Gibson. Disk failures in the real world: What does an MTTF of 1,000,000 mean to you? In Conference on File and Storage Technologies, [45] M. Sivathanu, V. Prabhakaran, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Improving storage system availability with D-GRAID. In Conference on File and Storage Technologies, [46] Apache Software. Pigmix. https://cwiki. apache.org/confluence/display/pig/ PigMix, [47] A. Thusoo, D. Borthakur, R. Murthy, Z. Shao, N. Jain, H. Liu, S. Anthony, and J. S. Sarma. Data warehousing and analytics infrastructure at Facebook. In SIGMOD, [48] L. Tian, D. Feng, H. Jiang, K. Zhou, L. Zeng, J. Chen, Z. Wang, and Z. Song. PRO: a popularity-based multi-threaded reconstruction optimization for RAIDstructured storage systems. In Conference on File and Storage Technologies, [49] Z. Wang, A. G. Dimakis, and J. Bruck. Rebuilding for array codes in distributed storage systems. CoRR, abs/ , [50] H. Weatherspoon and J. Kubiatowicz. Erasure coding vs. replication: A quantitative comparison. In Workshop on Peer-to-Peer Systems, [51] L. Xiang, Y. Xu, J. C. S. Lui, and Q. Chang. Optimal recovery of single disk failure in RDP code storage systems. In ACM SIGMETRICS, [52] L. Xu and J. Bruck. X-Code: MDS array codes with optimal encoding. IEEE Transactions on Information Theory, 45(1): , January [53] Z. Zhang, A. Deshpande, X. Ma, E. Thereska, and D. Narayanan. Does erasure coding have a role to play in my data center? Microsoft Technical Report MSR- TR ,

### Distributed Storage Networks and Computer Forensics

Distributed Storage Networks 5 Raid-6 Encoding Technical Faculty Winter Semester 2011/12 RAID Redundant Array of Independent Disks Patterson, Gibson, Katz, A Case for Redundant Array of Inexpensive Disks,

### Algorithms and Methods for Distributed Storage Networks 5 Raid-6 Encoding Christian Schindelhauer

Algorithms and Methods for Distributed Storage Networks 5 Raid-6 Encoding Institut für Informatik Wintersemester 2007/08 RAID Redundant Array of Independent Disks Patterson, Gibson, Katz, A Case for Redundant

### Towards High Security and Fault Tolerant Dispersed Storage System with Optimized Information Dispersal Algorithm

Towards High Security and Fault Tolerant Dispersed Storage System with Optimized Information Dispersal Algorithm I Hrishikesh Lahkar, II Manjunath C R I,II Jain University, School of Engineering and Technology,

### A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications

A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications Catherine D. Schuman James S. Plank Technical Report UT-CS-8-625 Department of Electrical Engineering and Computer

### A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster

A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster K. V. Rashmi 1, Nihar B. Shah 1, Dikang Gu 2, Hairong Kuang

### Theoretical Aspects of Storage Systems Autumn 2009

Theoretical Aspects of Storage Systems Autumn 2009 Chapter 2: Double Disk Failures André Brinkmann Data Corruption in the Storage Stack What are Latent Sector Errors What is Silent Data Corruption Checksum

### DiskReduce: RAID for Data-Intensive Scalable Computing

DiskReduce: RAID for Data-Intensive Scalable Computing Bin Fan binfan@cs.cmu.edu Wittawat Tantisiriroj wtantisi@cs.cmu.edu Garth Gibson garth@cs.cmu.edu Lin Xiao lxiao@cs.cmu.edu ABSTRACT Data-intensive

### Non-Redundant (RAID Level 0)

There are many types of RAID and some of the important ones are introduced below: Non-Redundant (RAID Level 0) A non-redundant disk array, or RAID level 0, has the lowest cost of any RAID organization

### Erasure Coding for Cloud Communication and Storage

Erasure Coding for Cloud Communication and Storage Cheng Huang, Jin Li Microsoft Research Tutorial at IEEE ICC (June, 2014) 1 Tutorial at IEEE ICC (June, 2014) 2 Tutorial at IEEE ICC (June, 2014) 3 Erasure

### A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries For Storage

A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries For Storage James S. Plank University of Tennessee plank@cs.utk.edu Lihao Xu Wayne State University Jianqiang Luo Wayne

### Data Storage - II: Efficient Usage & Errors

Data Storage - II: Efficient Usage & Errors Week 10, Spring 2005 Updated by M. Naci Akkøk, 27.02.2004, 03.03.2005 based upon slides by Pål Halvorsen, 12.3.2002. Contains slides from: Hector Garcia-Molina

### Overview. How distributed file systems work Part 1: The code repair problem Regenerating Codes. Part 2: Locally Repairable Codes

Overview How distributed file systems work Part 1: The code repair problem Regenerating Codes Part 2: Locally Repairable Codes Part 3: Availability of Codes Open problems 1 current hadoop architecture

### Definition of RAID Levels

RAID The basic idea of RAID (Redundant Array of Independent Disks) is to combine multiple inexpensive disk drives into an array of disk drives to obtain performance, capacity and reliability that exceeds

### Note: Correction to the 1997 Tutorial on Reed-Solomon Coding

Note: Correction to the 1997 utorial on Reed-Solomon Coding James S Plank Ying Ding University of ennessee Knoxville, N 3799 [plank,ying]@csutkedu echnical Report U-CS-03-504 Department of Computer Science

### Erasure Coding in Windows Azure Storage

Erasure Coding in Windows Azure Storage Cheng Huang Microsoft Corporation Joint work with Huseyin Simitci, Yikang Xu, Aaron Ogus, Brad Calder, Parikshit Gopalan, Jin Li, and Sergey Yekhanin Outline Introduction

### Data Storage - II: Efficient Usage & Errors. Contains slides from: Naci Akkök, Hector Garcia-Molina, Pål Halvorsen, Ketil Lund

Data Storage - II: Efficient Usage & Errors Contains slides from: Naci Akkök, Hector Garcia-Molina, Pål Halvorsen, Ketil Lund Overview Efficient storage usage Disk errors Error recovery INF3 7.3.26 Ellen

### Designing a Cloud Storage System

Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes

### A Reed-Solomon Code for Disk Storage, and Efficient Recovery Computations for Erasure-Coded Disk Storage

A Reed-Solomon Code for Disk Storage, and Efficient Recovery Computations for Erasure-Coded Disk Storage M. S. MANASSE Microsoft Research, Mountain View, California, USA C. A. THEKKATH Microsoft Research,

### Exploring RAID Configurations

Exploring RAID Configurations J. Ryan Fishel Florida State University August 6, 2008 Abstract To address the limits of today s slow mechanical disks, we explored a number of data layouts to improve RAID

### DELL RAID PRIMER DELL PERC RAID CONTROLLERS. Joe H. Trickey III. Dell Storage RAID Product Marketing. John Seward. Dell Storage RAID Engineering

DELL RAID PRIMER DELL PERC RAID CONTROLLERS Joe H. Trickey III Dell Storage RAID Product Marketing John Seward Dell Storage RAID Engineering http://www.dell.com/content/topics/topic.aspx/global/products/pvaul/top

### RAID Triple Parity. Peter Corbett. Atul Goel ABSTRACT. Categories and Subject Descriptors. General Terms. Keywords

RAID Triple Parity Atul Goel NetApp Inc atul.goel@netapp.com Peter Corbett NetApp Inc peter.corbett@netapp.com ABSTRACT RAID triple parity (RTP) is a new algorithm for protecting against three-disk failures.

### Enable and Optimize Erasure Code for Big data on Hadoop. High Performance Computing, Intel Jun Jin

Enable and Optimize Erasure Code for Big data on Hadoop High Performance Computing, Intel Jun Jin (jun.i.jin@intel.com) Agenda Background and overview Codec performance Block placement Policy Performance

### arxiv: v1 [cs.it] 16 Dec 2013

On the Effectiveness of Polynomial Realization of Reed-Solomon Codes for Storage Systems arxiv:555v [csit] 6 Dec Abstract Kyumars Sheykh Esmaili Technicolor Research Lab Paris, France kyumarssheykhesmaili@technicolorcom

### CSE-E5430 Scalable Cloud Computing P Lecture 5

CSE-E5430 Scalable Cloud Computing P Lecture 5 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 12.10-2015 1/34 Fault Tolerance Strategies for Storage

### Cloud Scale Storage Systems. Sean Ogden October 30, 2013

Cloud Scale Storage Systems Sean Ogden October 30, 2013 Evolution P2P routing/dhts (Chord, CAN, Pastry, etc.) P2P Storage (Pond, Antiquity) Storing Greg's baby pictures on machines of untrusted strangers

### A Cost-based Heterogeneous Recovery Scheme for Distributed Storage Systems with RAID-6 Codes

A Cost-based Heterogeneous Recovery Scheme for Distributed Storage Systems with RAID-6 Codes Yunfeng Zhu, Patrick P. C. Lee, Liping Xiang, Yinlong Xu, Lingling Gao School of Computer Science and Technology,

### EMC CLARiiON RAID 6 Technology A Detailed Review

A Detailed Review Abstract This white paper discusses the EMC CLARiiON RAID 6 implementation available in FLARE 26 and later, including an overview of RAID 6 and the CLARiiON-specific implementation, when

### The mathematics of RAID-6

The mathematics of RAID-6 H. Peter Anvin First version 20 January 2004 Last updated 20 December 2011 RAID-6 supports losing any two drives. syndromes, generally referred P and Q. The way

A Piggybacing Design Framewor for Read-and Download-efficient Distributed Storage Codes K V Rashmi, Nihar B Shah, Kannan Ramchandran, Fellow, IEEE Department of Electrical Engineering and Computer Sciences

### RAID-DP: NetApp Implementation of Double- Parity RAID for Data Protection

Technical Report RAID-DP: NetApp Implementation of Double- Parity RAID for Data Protection Jay White & Chris Lueth, NetApp May 2010 TR-3298 ABSTRACT This document provides an in-depth overview of the NetApp

### STAR : An Efficient Coding Scheme for Correcting Triple Storage Node Failures

STAR : An Efficient Coding Scheme for Correcting Triple Storage Node Failures Cheng Huang, Member, IEEE, and Lihao Xu, Senior Member, IEEE Abstract Proper data placement schemes based on erasure correcting

### 200 Chapter 7. (This observation is reinforced and elaborated in Exercises 7.5 and 7.6, and the reader is urged to work through them.

200 Chapter 7 (This observation is reinforced and elaborated in Exercises 7.5 and 7.6, and the reader is urged to work through them.) 7.2 RAID Disks are potential bottlenecks for system performance and

### HP Smart Array Controllers and basic RAID performance factors

Technical white paper HP Smart Array Controllers and basic RAID performance factors Technology brief Table of contents Abstract 2 Benefits of drive arrays 2 Factors that affect performance 2 HP Smart Array

### Evaluation of Object Placement Techniques in a Policy-Managed Storage System

Evaluation of Object Placement Techniques in a Policy-Managed Storage System Pawan Goyal Peter Radkov and Prashant Shenoy Storage Systems Department, Department of Computer Science, IBM Almaden Research

### Three-Dimensional Redundancy Codes for Archival Storage

Three-Dimensional Redundancy Codes for Archival Storage Jehan-François Pâris Darrell D. E. Long Witold Litwin Department of Computer Science University of Houston Houston, T, USA jfparis@uh.edu Department

### An Introduction to Galois Fields and Reed-Solomon Coding

An Introduction to Galois Fields and Reed-Solomon Coding James Westall James Martin School of Computing Clemson University Clemson, SC 29634-1906 October 4, 2010 1 Fields A field is a set of elements on

### A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems

A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems James S Plank Department of Computer Science University of Tennessee February 19, 1999 Abstract It is well-known that Reed-Solomon

### RAID Basics Training Guide

RAID Basics Training Guide Discover a Higher Level of Performance RAID matters. Rely on Intel RAID. Table of Contents 1. What is RAID? 2. RAID Levels RAID 0 RAID 1 RAID 5 RAID 6 RAID 10 RAID 0+1 RAID 1E

### A Hitchhiker s Guide to Fast and Efficient Data Reconstruction in Erasure-coded Data Centers

A Hitchhiker s Guide to Fast and Efficient Data Reconstruction in Erasure-coded Data Centers K V Rashmi 1, Nihar B Shah 1, Dikang Gu 2, Hairong Kuang 2, Dhruba Borthakur 2, and Kannan Ramchandran 1 ABSTRACT

### H-Code: A Hybrid MDS Array Code to Optimize Partial Stripe Writes in RAID-6

H-Code: A Hybrid MDS Array Code to Optimize Partial Stripe Writes in RAID-6 Chentao Wu 1, Shenggang Wan 2, Xubin He 1, Qiang Cao 2, and Changsheng Xie 2 1 Department of Electrical & Computer Engineering,

### The mathematics of RAID-6

The mathematics of RAID-6 H. Peter Anvin 1 December 2004 RAID-6 supports losing any two drives. The way this is done is by computing two syndromes, generally referred P and Q. 1 A quick

### Functional-Repair-by-Transfer Regenerating Codes

Functional-Repair-by-Transfer Regenerating Codes Kenneth W Shum and Yuchong Hu Abstract In a distributed storage system a data file is distributed to several storage nodes such that the original file can

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### PIONEER RESEARCH & DEVELOPMENT GROUP

SURVEY ON RAID Aishwarya Airen 1, Aarsh Pandit 2, Anshul Sogani 3 1,2,3 A.I.T.R, Indore. Abstract RAID stands for Redundant Array of Independent Disk that is a concept which provides an efficient way for

### RAID Overview: Identifying What RAID Levels Best Meet Customer Needs. Diamond Series RAID Storage Array

ATTO Technology, Inc. Corporate Headquarters 155 Crosspoint Parkway Amherst, NY 14068 Phone: 716-691-1999 Fax: 716-691-9353 www.attotech.com sales@attotech.com RAID Overview: Identifying What RAID Levels

### Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

### Lenovo RAID Introduction Reference Information

Lenovo RAID Introduction Reference Information Using a Redundant Array of Independent Disks (RAID) to store data remains one of the most common and cost-efficient methods to increase server's storage performance,

### Performance Report Modular RAID for PRIMERGY

Performance Report Modular RAID for PRIMERGY Version 1.1 March 2008 Pages 15 Abstract This technical documentation is designed for persons, who deal with the selection of RAID technologies and RAID controllers

### Reliability and Fault Tolerance in Storage

Reliability and Fault Tolerance in Storage Dalit Naor/ Dima Sotnikov IBM Haifa Research Storage Systems 1 Advanced Topics on Storage Systems - Spring 2014, Tel-Aviv University http://www.eng.tau.ac.il/semcom

### Magnus: Peer to Peer Backup System

Magnus: Peer to Peer Backup System Naveen Gattu, Richard Huang, John Lynn, Huaxia Xia Department of Computer Science University of California, San Diego Abstract Magnus is a peer-to-peer backup system

### Energy aware RAID Configuration for Large Storage Systems

Energy aware RAID Configuration for Large Storage Systems Norifumi Nishikawa norifumi@tkl.iis.u-tokyo.ac.jp Miyuki Nakano miyuki@tkl.iis.u-tokyo.ac.jp Masaru Kitsuregawa kitsure@tkl.iis.u-tokyo.ac.jp Abstract

### Click on the diagram to see RAID 0 in action

Click on the diagram to see RAID 0 in action RAID Level 0 requires a minimum of 2 drives to implement RAID 0 implements a striped disk array, the data is broken down into blocks and each block is written

### Efficient LDPC Code Based Secret Sharing Schemes and Private Data Storage in Cloud without Encryption

Efficient LDPC Code Based Secret Sharing Schemes and Private Data Storage in Cloud without Encryption Yongge Wang Department of SIS, UNC Charlotte, USA yonwang@uncc.edu Abstract. LDPC codes, LT codes,

### an analysis of RAID 5DP

an analysis of RAID 5DP a qualitative and quantitative comparison of RAID levels and data protection hp white paper for information about the va 7000 series and periodic updates to this white paper see

### Reliability of Data Storage Systems

Zurich Research Laboratory Ilias Iliadis April 2, 25 Keynote NexComm 25 www.zurich.ibm.com 25 IBM Corporation Long-term Storage of Increasing Amount of Information An increasing amount of information is

### RAID: Redundant Arrays of Inexpensive Disks this discussion is based on the paper: on Management of Data (Chicago, IL), pp.109--116, 1988.

: Redundant Arrays of Inexpensive Disks this discussion is based on the paper:» A Case for Redundant Arrays of Inexpensive Disks (),» David A Patterson, Garth Gibson, and Randy H Katz,» In Proceedings

### Comparing Dynamic Disk Pools (DDP) with RAID-6 using IOR

Comparing Dynamic Disk Pools (DDP) with RAID-6 using IOR December, 2012 Peter McGonigal petermc@sgi.com Abstract Dynamic Disk Pools (DDP) offer an exciting new approach to traditional RAID sets by substantially

### Filing Systems. Filing Systems

Filing Systems At the outset we identified long-term storage as desirable characteristic of an OS. EG: On-line storage for an MIS. Convenience of not having to re-write programs. Sharing of data in an

### Benchmarking Hadoop & HBase on Violin

Technical White Paper Report Technical Report Benchmarking Hadoop & HBase on Violin Harnessing Big Data Analytics at the Speed of Memory Version 1.0 Abstract The purpose of benchmarking is to show advantages

### IBM ^ xseries ServeRAID Technology

IBM ^ xseries ServeRAID Technology Reliability through RAID technology Executive Summary: t long ago, business-critical computing on industry-standard platforms was unheard of. Proprietary systems were

### RevoScaleR Speed and Scalability

EXECUTIVE WHITE PAPER RevoScaleR Speed and Scalability By Lee Edlefsen Ph.D., Chief Scientist, Revolution Analytics Abstract RevoScaleR, the Big Data predictive analytics library included with Revolution

### RAID Performance Analysis

RAID Performance Analysis We have six 500 GB disks with 8 ms average seek time. They rotate at 7200 RPM and have a transfer rate of 20 MB/sec. The minimum unit of transfer to each disk is a 512 byte sector.

### Disk Array Data Organizations and RAID

Guest Lecture for 15-440 Disk Array Data Organizations and RAID October 2010, Greg Ganger 1 Plan for today Why have multiple disks? Storage capacity, performance capacity, reliability Load distribution

### RAID 5 rebuild performance in ProLiant

RAID 5 rebuild performance in ProLiant technology brief Abstract... 2 Overview of the RAID 5 rebuild process... 2 Estimating the mean-time-to-failure (MTTF)... 3 Factors affecting RAID 5 array rebuild

### Impact of Stripe Unit Size on Performance and Endurance of SSD-Based RAID Arrays

1 Impact of Stripe Unit Size on Performance and Endurance of SSD-Based RAID Arrays Farzaneh Rajaei Salmasi Hossein Asadi Majid GhasemiGol rajaei@ce.sharif.edu asadi@sharif.edu ghasemigol@ce.sharif.edu

### Practical Data Integrity Protection in Network-Coded Cloud Storage

Practical Data Integrity Protection in Network-Coded Cloud Storage Henry C. H. Chen Department of Computer Science and Engineering The Chinese University of Hong Kong Outline Introduction FMSR in NCCloud

### Disk Storage & Dependability

Disk Storage & Dependability Computer Organization Architectures for Embedded Computing Wednesday 19 November 14 Many slides adapted from: Computer Organization and Design, Patterson & Hennessy 4th Edition,

### CONFIGURATION GUIDELINES: EMC STORAGE FOR PHYSICAL SECURITY

White Paper CONFIGURATION GUIDELINES: EMC STORAGE FOR PHYSICAL SECURITY DVTel Latitude NVMS performance using EMC Isilon storage arrays Correct sizing for storage in a DVTel Latitude physical security

### Online Code Rate Adaptation in Cloud Storage Systems with Multiple Erasure Codes

Online Code Rate Adaptation in Cloud Storage Systems with Multiple Erasure Codes Rui Zhu 1, Di Niu 1, Zongpeng Li 2 1 Department of Electrical and Computer Engineering, University of Alberta, Edmonton,

### SSDs and RAID: What s the right strategy. Paul Goodwin VP Product Development Avant Technology

SSDs and RAID: What s the right strategy Paul Goodwin VP Product Development Avant Technology SSDs and RAID: What s the right strategy Flash Overview SSD Overview RAID overview Thoughts about Raid Strategies

### RAID-DP : NETWORK APPLIANCE IMPLEMENTATION OF RAID DOUBLE PARITY FOR DATA PROTECTION A HIGH-SPEED IMPLEMENTATION OF RAID 6

RAID-DP : NETWORK APPLIANCE IMPLEMENTATION OF RAID DOUBLE PARITY FOR DATA PROTECTION A HIGH-SPEED IMPLEMENTATION OF RAID 6 Chris Lueth, Network Appliance, Inc. TR-3298 [12/2006] ABSTRACT To date, RAID

### Surviving Two Disk Failures. Introducing Various RAID 6 Implementations

Surviving Two Failures Introducing Various RAID 6 Implementations Notices The information in this document is subject to change without notice. While every effort has been made to ensure that all information

### systems' resilience to disk failure through

BY Shashwath Veerappa Devaru CS615 Aspects of System Administration Using Multiple Hard Drives for Performance and Reliability RAID is the term used to describe a storage systems' resilience to disk failure

### A Study on Workload Imbalance Issues in Data Intensive Distributed Computing

A Study on Workload Imbalance Issues in Data Intensive Distributed Computing Sven Groot 1, Kazuo Goda 1, and Masaru Kitsuregawa 1 University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan Abstract.

### Virtuoso and Database Scalability

Virtuoso and Database Scalability By Orri Erling Table of Contents Abstract Metrics Results Transaction Throughput Initializing 40 warehouses Serial Read Test Conditions Analysis Working Set Effect of

### Comprehending the Tradeoffs between Deploying Oracle Database on RAID 5 and RAID 10 Storage Configurations. Database Solutions Engineering

Comprehending the Tradeoffs between Deploying Oracle Database on RAID 5 and RAID 10 Storage Configurations A Dell Technical White Paper Database Solutions Engineering By Sudhansu Sekhar and Raghunatha

### A Tale of Two Erasure Codes in HDFS

A Tale of Two Erasure Codes in HDFS Mingyuan Xia, Mcill University; Mohit Saxena, Mario Blaum, and David A. ease, IBM Research Almaden https://www.usenix.org/conference/fast15/technical-sessions/presentation/xia

### Summer Student Project Report

Summer Student Project Report Dimitris Kalimeris National and Kapodistrian University of Athens June September 2014 Abstract This report will outline two projects that were done as part of a three months

### How to recover a failed Storage Spaces

www.storage-spaces-recovery.com How to recover a failed Storage Spaces ReclaiMe Storage Spaces Recovery User Manual 2013 www.storage-spaces-recovery.com Contents Overview... 4 Storage Spaces concepts and

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

Nutanix Tech Note Failure Analysis A Failure Analysis of Storage System Architectures Nutanix Scale-out v. Legacy Designs Types of data to be protected Any examination of storage system failure scenarios

### RAID 0 (striping) RAID 1 (mirroring)

RAID 0 (striping) RAID 0 uses the read/write capabilities of two or more hard drives working together to maximize storage performance. Data in a RAID 0 volume is arranged into blocks that are spread across

### File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System

CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID

### Distribution One Server Requirements

Distribution One Server Requirements Introduction Welcome to the Hardware Configuration Guide. The goal of this guide is to provide a practical approach to sizing your Distribution One application and

Copyright www.agileload.com 1 INTRODUCTION Performance testing is a complex activity where dozens of factors contribute to its success and effective usage of all those factors is necessary to get the accurate

### Compact Representations and Approximations for Compuation in Games

Compact Representations and Approximations for Compuation in Games Kevin Swersky April 23, 2008 Abstract Compact representations have recently been developed as a way of both encoding the strategic interactions

### DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies

### Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs

Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs Kevin M. Greenan ParaScale greenan@parascale.com Xiaozhou Li HP Labs xiaozhou.li@hp.com Jay J. Wylie HP

### Remote Data Integrity Checking for Public Clouds

IBM Student Workshop for Frontiers of Cloud Computing 01 Remote Data Integrity Checking for Public Clouds Bo (Irvine) Chen Department of Computer Science New Jersey Institute of Technology joint work with

### XtremIO DATA PROTECTION (XDP)

White Paper XtremIO DATA PROTECTION (XDP) Flash-Specific Data Protection, Provided by XtremIO (Ver..0) Abstract This white paper introduces the and discusses its benefits and advantages over RAID, with

### GENERAL INFORMATION COPYRIGHT... 3 NOTICES... 3 XD5 PRECAUTIONS... 3 INTRODUCTION... 4 FEATURES... 4 SYSTEM REQUIREMENT... 4

1 Table of Contents GENERAL INFORMATION COPYRIGHT... 3 NOTICES... 3 XD5 PRECAUTIONS... 3 INTRODUCTION... 4 FEATURES... 4 SYSTEM REQUIREMENT... 4 XD5 FAMILULARIZATION... 5 PACKAGE CONTENTS... 5 HARDWARE

### Replication and Erasure Coding Explained

Replication and Erasure Coding Explained Caringo Elastic Content Protection Technical Overview Paul Carpentier, CTO, Caringo Introduction: data loss Unlike what most storage vendors will try to make you

### Network Coding for Distributed Storage

Network Coding for Distributed Storage Alex Dimakis USC Overview Motivation Data centers Mobile distributed storage for D2D Specific storage problems Fundamental tradeoff between repair communication and

### Evaluating HDFS I/O Performance on Virtualized Systems

Evaluating HDFS I/O Performance on Virtualized Systems Xin Tang xtang@cs.wisc.edu University of Wisconsin-Madison Department of Computer Sciences Abstract Hadoop as a Service (HaaS) has received increasing

### Copysets: Reducing the Frequency of Data Loss in Cloud Storage

Copysets: Reducing the Frequency of Data Loss in Cloud Storage Asaf Cidon, Stephen Rumble, Ryan Stutsman, Sachin Katti, John Ousterhout and Mendel Rosenblum Stanford University cidon@stanford.edu, {rumble,stutsman,skatti,ouster,mendel}@cs.stanford.edu

### Index Terms Cloud Storage Services, data integrity, dependable distributed storage, data dynamics, Cloud Computing.

Volume 3, Issue 5, May 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Privacy - Preserving

### Load Balancing in Fault Tolerant Video Server

Load Balancing in Fault Tolerant Video Server # D. N. Sujatha*, Girish K*, Rashmi B*, Venugopal K. R*, L. M. Patnaik** *Department of Computer Science and Engineering University Visvesvaraya College of

### Windows Server Performance Monitoring

Spot server problems before they are noticed The system s really slow today! How often have you heard that? Finding the solution isn t so easy. The obvious questions to ask are why is it running slowly