The Stragegy for Fat I/O in Degrae Moe of RAID-5 *DONG-JAE, KANG *CHANG-SOO, KIM **BUM-JOO,SHIN *Comuter & Sytem Lab. ETRI 161 Gajeong-Dong, Yueong-Gu, Deajeon, 35-35 KOERA **125-1 Naei-Dong Miryang Gyeongnam KOREA Abtract: - RAID ha been ue for high erformance an high availability of ata, but it ha roblem that cot of ata regeneration for uorting high availability in ik failure moe(egrae moe) an cot of ata rebuiling for recovery of faile ik are very exenive. In thi aer, we uoe the tragegy for fat I/O in egrae moe of RAID5. For rooe metho, we a a are ik to traitional RAID5 architecture, an ue it for getting better erformance in egrae moe an reucing the cot at ik recovery time. When I/O requet, rea an write oeration, occurre for block in faile ik, requet block can be regenerate correctly by reaing correoning block from urviving member ik an comuting the excluive OR of their content. An regenerate block i ave aitional are ik. then, mark recovery-information to S.D(Sare Dik) recovery bitma. Requet after thi roce can ue the block recovere in are ik an it i one reeatly. Thi trategy allow ata to be rea or written with normal erformance by acceing to are ik block, not faile ik block. an require only non-accee ata in egrae moe to be recovere on rebuiling time. By uing uoe trategy, we have everal avantage, better erformance in egrae moe of RAID5,low cot at ik recovery time an hort reone time. Key-Wor: - RAID5, Degrae moe, I/O erformance, Sare ik, Rebuiling, Regeneration 1 Introuction RAID imrove I/O erformance becaue they ten to balance tream of either equential or ranom I/O requet aroximately acro the member ik an rogre ata availability by toring reunant ata that allow uer ata to be regenerate if the ik on which it i tore fail. Aitionally, it imlify torage management by aggregating multile hyical ik into one large virtual ik an treating more torage caacity a a ingle management entity. RAID5 mot frequently ue to minimize the write erformance bottleneck an uort fault-tolerance by the itribution of ucceive arity trie acro ome or all ik member cyclically. But it ha roblem that cot of ata regeneration for offering high availability in ik failure moe(egrae moe) an cot of ata rebuiling for recovery of faile ik are very exenive. Degrae moe refer to the ituation that in the cae of one ik failure an no attemt ha been mae to rebuil the ta to lace it on anther ik, ytem can till work without interrute[7]. But it ha a ignificant increae in the ytem loa when ingle ik i faile, egrae moe. Becaue rea oeration for the faile ik caue n-1 rea accee to the relative block on urviving ik to regenerate lot ata block. An write oeration for the faile ik caue n-1 rea accee to the relative block on urviving member ik to comute the arity block an one write acce to write the comute arity information to the block of the correoning ik. Like uer ecrition, RAID5 ha eriou efect in egrae moe. In thi aer, we uoe the trategy for fat I/O in egrae moe an reuce recovery cot at ata rebuiling of faile ik 2 Relate Work When ingle ik failure i occurre in RAID5, faile ik houl be relace with a new ik to rebuil ata. Otherwie, RAID5 erformance will be very low own to regenerate ata uing the urviving member ik in triing et for every rea / write oeration[1]. For the at everal year, many reearche for metho to imrove egrae erformance of RAID5 have been roceee. When all ik in an array are oerational, it i calle to be normal moe, the array i ai to be in egrae
moe when ingle ik in the RAID5 ha faile an no attemt ha been mae to rebuil the ata that ue to be on that ik an lace it on another ik. An the array i ai to be in rebuil moe uring the time that ata on the faile ik i being rebuilt an lace on another ik[7]. To recover the ata in faile ik, RAID ue are ik cheme, eicate aring cheme, itribute aring cheme an arity aring cheme. Deicate aring i o that ata on faile ik can be immeiately rebuilt to a are rive. An itribute aring ha the itribute are ace among the ik in a RAID, the arity aring i milar to the itribute aring but it ha not are ace,when ingle ik i faile, trie unit in ifferent ik are combine to form larger trie et an ik[2][7]. The itribute aring an arity aring imrove the ata write bottleneck, when faile ata i recover. Sare ik cheme ecribe uer are recovery metho for ata in faile ik, an they on't guarantee I/O erformance in egrae moe. But there i many metho to imrove erformance in egrae moe[5][7]. If ingle ik i faile, arity block in RAID5 i change to normal ata block an it i ue a ace for ata of faile ik. The metho can uort erformance in normal moe. But when ata in faile ik i rebuil on recovery time, till the cot of recovery that ata i regenerate by excluive-or an written to new ik i very exenive. 3 The Strategy for Fat I/O Fig.1 i the architecture which i for uoe trategy in thi aer. For reente trategy, are ik i ae to traitional RAID5 architecture, an it ha a S.D(Sare Dik) Recovery Bitma to check whether a block in faile ik i recovere or not. RAID-5 trie unit arity unit ata unit S.D Recovery Bitma Fig.1 Architecture for uoe metho Sare Dik Firt, let' efine terminologie ue in thi aer. Data unit mean the unit of ata acce uorte by the array, an arity unit rereent reunancy information generate from the bitwie excluive-or of the collection of ata unit. The ize of ata unit an arity unit i ame. The reunancy grou forme by ata unit an arity unit i calle trie unit. Suoe metho i for imroving I/O erformance in egrae moe. So, in thi aer, we ecribe only I/O trategy in ingle ik failure moe, egrae moe. 3.1 Rea Strategy In thi ectrion, we ecribe rea trategy in the egrae moe of RAID5. 4 7 3 READ requet 1 5 2 1 2 1 8 11 6 9 12 ik ik1 ik2 ik3 faile ik S.D recovery Bitma / 1 1 2 3 Sare ik Fig.2 Rea oeration in egrae moe of RAID5 Fig.2 reent the metho of rea oeration in egrae moe, in ytem aate by uoe trategy. Let' aume ik1 i faile to exlain rea oeration in egrae moe. If a requet for ata block, 1, in faile ik i occurre, S.D Recovery Bitma i invetigate to enure whether the faile block i recovere or not before rea oeration i execute. Bitma i marke a "not recovere", rea oeration that ma to faile ik regenerate requete ata from urviving ik member a illutrate in [Fig. 2]. To regenerate the block, it rea ata unit(, 2) an arity unit(), then execute excluive-or among rea ata. The reult i returne to uer alication. Aitionally it i ave on block() in are ik inclue to ame trie unit. If it i finihe, the correoning bitma(s.d Recovery Bitma) i marke a "recovere". Otherwie, if bitma i checke a "recovere" on rea oeration,, it mean re-acce to the block after ingle ik i faile. In thi cae, rea oeration that ma to the block in faile ik rea the ata from ame block in are ik, without the cot of regeneration for faile ata. So, re-acce
requet for ata in faile ik i rocee in normal moe erformance. It ha everal avantage, no nee of regeneration cot for faile block, revention from uen fall of erformance, etc. An it imrove the roblem in egrae moe of RAID5. 3.2 Write Strategy In thi chater, we ecribe write trategy in the egrae moe of RAID5. Fig.2 reent the metho of write oeration in egrae moe, in ytem aate by uoe trategy. Let u aume that ik1 i failure to ecribe write oeration in egrae moe, an a requet for ata(5) in faile ik i occurre. WRITE requet New Block S.D recovery Bitma oeration. In next rea oeration, the ata block recovere in write oeration time can i ue, too. It uort normal moe erformance in egrae moe by amitting inut an outut from/to are ik an imrove elaye reone time an uen fall of ytem erformance in egrae moe. 3.3 Data Rebuiling Strategy Fig.4 reent rebuiling roce of ata in faile ik after aating uoe cheme., 1 an 4 i re-accee block by rea an write oeration in egrae moe, an it ha recovere block for faile ik. If a block in faile ik i accee by I/O oeration, when the block i re-accee, it i recovere on ame block in are ik, an it can be erve like normal ik tate. S.D Recovery Bitma 4 7 3 1 5 2 1 faile ik 2 1 8 11 6 9 12 / 1 1 / 5 2 3 Sare ik Fig.3 Write oeration in egrae moe of RAID5 Like rea oeration, write oeration invetigate S.D Recovery Bitma before it i execute wether it i marke a "recovere" or not. In cae that checke a "not recovere", it rea ata(4, 6) from member ik in trie unit to uate correoning arity ata(1). If it i comlete, then execute excluive-or oeration between rea ata an buffer ata to be written to faile ik block. The reult i ientical with 5 ata an it i ave 1 to uate the arity ata. Aitionally, it i ave to 1 block in are ik to recover the faile ik block. Finally, it mark the correoning S.D recovery bitma a "recovere". In cae that bitma i checke a "recovere", it mean re-acce to a block after the ik i faile. In thi cae, a i the ame with normal write oeration, rea the correoning ata block(4, 6) to uate arity block(1), an then execute excluive-or oeration. If it i comlete, uate the arity block, an requete ata block in are ik. In cae requete block in faile ik i re-acce block, rea oeration for 4, 6 ata block i not nee. So, more econ acce for ame block in faile ik i ientical with normal moe ik ik1 ik2 ik3 4 7 3 13 1 5 2 1 14 2 1 8 11 15 6 9 12 / 1 1 / 5 2 3 4 / 14 16 17 5 18 5 faile ik Sare ik 4 1 2 3 4 5 Fig.4 Rebuiling faile ik in uoe Strategy If I/O oeration are rocee for faile ik in egrae moe, more ata block for faile ik are recovere in are ik. So the erformance of the ytem aate by uoe trategy aroximate to normal moe erformance, an the cot of I/O in egrae moe i reuce. Rebuiling or recontruction mean that make a relacement ik' content conitent with thoe of the remaining normal member ik. To o it, reaing correoning trie unit ata from each of the urviving original memver ik an comuting the excluive-or of thee trie unit' content, then writing the reult to relacement ik. Rebuiling i time-conuming job. It can lat u to everal hour for large ik. So RAID ytem mut be caable of oerating while it i occurring. Oeration while rebuiling roce ten to increae rebuil time, but rovie the comenating benefit of continuou ata availability to alication. In traitional RAID5 ytem, rebuiling i rocee through whole block in faile ik, it overhea i very eriou.
But, in reente trategy, we ha a avantage that can execute elective rebuiling for only not-accee block, not-recovere block in are ik, in egrae moe. In [Fig. 4],, 1 an 4 mean recovere block, after ingle ik, ik1, i faile. So, when ata rebuiling i neee to recover faile ik. Only 2, 3 an 5 i electe. Rebuiling roce in uoe trategy i the ame with traitional RAID5 ytem, excet uorting elective rebuiling, If elective rebuiling i comlete, are ik i relace with faile ik to make the RAID5 ytem a normal. Selective rebuiling ha low cot for ik recovery comaring of traitional ytem, an imrove uen fall of ytem erformance in recovery time. 3.4 Avantage of Suoe Strategy In thi ection, we ecribe everal avantage that aring ik cheme uort. Aume the number of ik coniting of RAID5 array i N. Firt, when rea oeration i rocee in egrae moe, regeneration for faile ik block alway require N-1 ik acce in traitional RAID5 ytem, reaing N-2 ata block an 1 arity block from correoning member ik. In uoe metho, firt rea oeration to a faile ik block require N ik acce, but rea oeration for re-accee faile ik block nee 1 ik acce, a rea oeration i rocee in normal moe. Secon, when write oeration i execute in egrae moe, write on faile ik block alway require N ik accee an bitwie oeration, reaing N-2 ata unit an 1 arity unit in trie unit, an excluive-or oeration among ata unit an arity unit, an 1 ik acce to write uate arity block to relative ik. In uoe metho, when firt write to a faile ik block require N+1 ik accee, but write oeration for re-accee faile ik block have the ame ik accee in comare with write of normal moe. Thir, when ik recovery, rebuiling, i rocee, traitional RAID ytem require recovery of whole block in faile ik. but, in uoe metho, require recovery of elective block, not accee block by rea or write oeration in egrae moe. Decribe avantage, uer, can imrove the uen fall of ytem erformance in egrae moe, an reuce the cot of rebuiling on recovery time. It guarantee that the ytem erformance in egrae moe aroximate to that of normal moe. 4. Evaluation an Analyi In thi ection, we reort the exerimental reult obtaine in uoe trategy. The exeriment were eigne to comare the erformance in between egrae moe an normal moe of RAID5 an to analyze cot of rebuiling roce. The environment for tet i ecribe in ection 4.1. 4.1 Environment for Envaluation We ue SANtoiaVM(SANtoia Volume Manager), logical volume manager, to tet uoe trategy, it ha been imlemente at Comuter & Sytem De. ETRI in Korea. Dik Array Switch HBA Hot Machine Fiber channel DataDataDik Fig.5 Environment for teting the trategy Table 1 Detaile environmental value for teting Env. element Secification Comuter Comaq Server(m) O.S Linux 6.2(kernel-2.2.18) RAM 512 Tet Sytem SANtoia Volume Manager Switch to StoragSAN(Fibre channel) Hot to Switch LAN Storage RAID-5(5M * 4) Strie unit 128K SANtoiaVM(SVM) reent the hyical torage a one or more virtual ik, volume, by converting I/O requet irecte to a virtual ik to I/O oeration on the unerlying member of hyical ik. it can imly the management of many torage. SVM uort variou RAID-level, Linear, triing, mirroring, an arity RAID. Fig. 5 rereent environment for tet. Whole ytem conit of hot machine, witch an torage grou. An they are connecte with fiber
channel or ethernet. The connection between hot machine an witch conit of LAN, ue to tranfer TCP/IP bae meage among the hot. An connection between witch an torage grou i fiber channel, ue to tranfer SCSI bae I/O ata. The etaile ec refer to Table 1. 4.2 Performance Evaluation We reent the erformance of rea / write oeration in egrae moe an in normal moe, an analyze it. Then, we comare two cae, to enure that uoe metho can imrove the erformance in egrae moe an reuce the cot of rebuiling. Each tet i erforme three time to obtain correct teting value. 4.2.1 Performance of Rea Oeration Fig.6 reent erformance of rea oeration in each moe, normal moe, egrae moe an S.D moe. In thi tet, egrae moe mean that of traitional RAID5 not-aating aring ik cheme. An S.D moe mean egrae moe aating uoe metho, aring ik cheme. Fig.6 how that I/O cot in egrae moe require three time more than normal moe. Rea oeraton in normal moe i atifie a 1 ik acce, but that in egrae moe require N-1 ik accee when RAID5 array conit of N-ik. Require Time( ec ) 12 1 8 6 4 2 Normal moe Failure moe S.D moe 1M 1M 1M 5M 1G I/O Block Size( 1KB / block ) Fig.6 Rea erformance in each moe That i, rea oeration in egrae moe rea ata unit an arity unit to regenerate the requete block. So, rea oeration in egrae moe ha roblem, uen fall of ytem erformance. But, Suoe metho imrove it by uorting I/O through aitional are ik. Suoe trategy regenerate the faile ik block an ave it in aring ik. Then, next rea oeration for the ame block in faile ik i erve by recovere block in are ik. It guarantee that erformance in egrae moe i imrove. Aitionally, on rebuiling time, it i ue to reuce the cot of recovery. In Fig.6, rea erformance grah for 1M ata in S.D moe ha low erformance than that of egrae moe, becaue of aitional write to aring ik. But a execution of rea oeration i continue, it aroximate to normal tate. 4.2.2 Performance of Write Oeration Fig.7 reent erformance of write oeration in each moe. Require Time( ec ) 12 1 8 6 4 2 Normal moe Failure moe S.D moe 1M 1M 1M 5M 1G I/O Block Size( 1KB / block ) Fig.7 Write erformance in each moe Like the grah of rea erformance, write erformance in S,D moe i lower than egrae moe for le 1M ata becaue of aitional write to are ik. But we have a goo erformance through one more ik acce. Re-accee for faile ik block can be rocee like a normal moe oeration by writing ata to are ik. The hae of grah can be effecte by trie unit ize, ata ize to be written, acce frequency for ame ata an etc. Many re-accee for ame ata make ytem ha a goo erformance. Acce Rate to the Same Block(%) Acce Rate to the Same Block 1 9 8 7 6 5 4 3 2 1 1M 1M 1M 5M 1G I/ O Block Size Fig.8 Acce Rate to the Same Block
Fig.8 how acce ratio to ame block accoring to increae the number of block to be written. It i tete for 5M caacity ik an other environmental value i ame. In cae 5M ata block i written, the re-acce ratio i about 37%, an that of 1G ata block i about 57%. So, if oeration in egrae moe i execute continuouly, re-acce ratio to ame block i increae. An the erformance in S.D moe i imrove becaue faile ik block i recovere in are ik an I/O requet for faile ik block i erve by are ik block. 4.2.3 Performance in Rebuiling moe Suoe trategy imrove erformance in egrae moe by recovering faile ik ata to are ik an uorting I/O through it. When ingle ik i faile in RAID, it i require to be relace a new normal ik. In thi aer, while uoe metho erform I/O requet, it rebuil the faile ik block. So, on recovery time, elective rebuiling i neee, not rebuiling of whole ata block in faile ik, it can reuce cot of recovery. Fig.9 how recovery ratio accoring to increaing the number of I/O block in egrae moe. We ue 5M caacity ik to tet recovery ratio. In cae I/O of 5M ata i erforme, the recovery ratio of faile ik aroach about 63%, an that of 1G ata block ha about 86% recovery ratio. So, by recovery time, if 5M ata block i alreay written or rea, Only 37% ata block of whole ik i neee to be rebuilt. Recovery Rate( % ) Dik Recovery Rate 1 9 8 7 6 5 4 3 2 1 1M 1M 1M 5M 1G I/O Block Size( 1KB / block ) Fig.9 Recovery ratio accoring to the number of I/O block In reente metho, if re-accee to a faile ik block i rare, I/O erformance in egrae moe will be ba. But recovery ratio for faile ik block will be high. While, if re-accee to a faile block i frequent, the reult will be revere. So I/O erformance in egrae moe an recovery ratio for faile ik block have a relationhi of invere roortion. Reorte reult will be change by many teting variable, that i, the number of array ik, trie unit ize, the caacity of ik, etc. 5. Concluion an Future Work RAID5 i ue for high erformance an high availability of ata, but it ha roblem that cot of ata regeneration for offering ata availability in egrae moe an of ata rebuiling for recovery of faile ik i very exenive. In thi aer, we uoe trategy to reolve the roblem an imrove the erformance in egrae moe. It uort the fat I/O roceing by recovering faile ik block to are ik an erving I/O requet throuth it. Aitionally, on recovery time, it can reuce the cot of recovery by uing ata block regenerate in are ik. Suoe metho can roce the I/O in egrae moe a like normal moe oeration after firt acce to the faile block. So it make the ytem to get better erformance in egrae moe. In future work, tet houl be erforme for variou environmental item, the number of array ik, variou trie unit ize, variou caacity of ik, etc. An the etaile reult i require to be reorte an analyze. Reference : [1] Paul Maiglia, "The RAID book" 6th eition, RAID Aviory Boar, 1997. [2] Sanghoon Jeon, Byoungchul Ahn, A Cot - Effective Solution to the Single ik Failure in RAID Architecture, Communication, Comuter an Signal Proceing, 1997,. 285-288 vol.1. [3] Hai Jin, Kai hwang, Jiangling Zhang, A RAID Reconfiguration Scheme for Gracefully Degrae Oeration, Parallel an Ditribute Proceing, 1999, PDP 99. Proceeing of the Seventh Euomicro Workho, 1999,.66-73. [4] M.Hollan, G,Gibon, an D.Siewiorek, Fat, On-line Failure Recovery in Reunant Dik Array, In 23r Annual International Symoium on Fault-Tolerant Comuting, 1993. [5] Mualaneni, N, Goinath, K, A multi-tier I/O RAID torage ytem with RAID1 an RAID5, Parallel an Ditribute Proceing Symoium, 2. IPDPS 2. Proceeing. 14th International, 2,. 663-671. [6] Davi A. Patteron, Garth A. Gibon, Rany H. Katz, A Cae for Reunant Array of Inexenive
Dik(RAID), ACM SIGMOD Conference Proceeing, 1988,. 19-116. [7] Jai Menon, Dick Matton, Comarion of Saring Alternative for Dik Array, Proceeing of International Symoium on Comuter Architecture, 1992,. 318-329. [8] Peter M. Chen an Ewar K. Lee, Striing in a RAID Level 5 Dik Array, In Proceeing of the Joint International Conference on Meaurement an Moeling of Comuter Sytem, 1995, 136-145. [9] Kai Hwang, Hai Jin, Roy Ho, RAID-x: A New Ditribute Dik Array for I/O-Centric Cluter Comuting, High-Performance Ditribute Comut ing, 2. Proceeing, 2, 279-286. [1] P. M. Chen, E. K. Lee, G. A. Gibon, R. H. Katz an D. A. Patteron, RAID: High- Performance, Reliable Seconary Storage, ACM Comuting Survey, Vol. 26, No.2, 1994, 145-185. [11] G. Gibon an D. Patteron, Deigning Dik Array for High Data Reliability, Journal of Parallel an Ditribute Comuting, January, 1993, 4-27 [12] Technology Forum, Lt., I RAID Five Level of Confuion?, Comuter Technology Revi ew, February, 1993, 37. [13] N. H. Vaiya, A Cae for Two-Level Ditribute Recovery Scheme, Proc. ACM International Conf. on Meaurement an Moeling of Comuter Sytem(Sigmetric '95), 64-73. [14] P. Cao, S. B. Lim, S. Venkataraman, an J. Wilke, The TickerTAIP Parallel RAID Architecture, ACM Tran. on Comuter Sytem, Vol.12, No.3, 1994, 236-296. [15] G. A. Alvarez, W. A. Burkhar, an F. Critian, Tolerating Multile Failure in RAID Architecture with Otimal Storage an Uniform Declutering, Proceeing of the 24th Annual ACM/IEEE International Symoium on Comuter Architecture, 1997, 62-72. [16] A. Thomaian, an J. Menon, RAID 5 Performance with Ditribute Saring, IEEE Tranaction on Parallel an Ditribute Sytem, Vol.8, No.6, 1997, 64-657. [17] M. Y. Lee, M. S. Park, Double Parity Saring for Performance Imrovement in Dik Array, Proceeing of the International Conference on Parallel an Ditribute Sytem, 1966.