A FASTER EXTERNAL SORTING ALGORITHM USING NO ADDITIONAL DISK SPACE

Transcription

1 47 A FASTER EXTERAL SORTIG ALGORITHM USIG O ADDITIOAL DISK SPACE Md. Rafqul Islam +, Mohd. oor Md. Sap ++, Md. Sumon Sarker +, Sk. Razbul Islam + + Computer Scence and Engneerng Dscplne, Khulna Unversty, Khulna-908, Bangladesh. ++ Faculty of Computer Scence and Informaton System, Unversty of Technology Malaysa, Skuda, Johor Bahru, Malaysa dmr978@yahoo.com, mohdnoor@fsksm.utm.my, sumonsrkr@yahoo.com, optmst_95@yahoo.com Abstract The effectve performance of the external sortng s analyzed n terms of both tme and I/O complextes. Ths paper s concerned wth a more effcent external sortng algorthm, where both the tme and I/O (read and wrte) complextes have been reduced. The proposed method s a hybrd technque that uses quck sort and merge sort n two dstnct phases. Both the tme and I/O complextes of the proposed algorthm are analyzed here and compared wth the complextes of the exstng smlar algorthms. The proposed algorthm uses specal n-place mergng technque, whch creates no extra backup fle for manpulatng huge records. For ths, the algorthm saves huge dsk space, whch s needed to hold the large fle. Ths also reduces tme complexty and makes the algorthm faster. Keywords: complexty, external sortng, n-place mergng, quck sort, runs.. ITRODUCTIO Sortng means to arrange records or data n an ascendng or descendng order. But when the sze of the records become so much larger than the nternal memory can hold at a tme, then the stuaton arse to use external sortng. It s qute dfferent nternal sortng, even though the problem n both cases s to sort a gven fle nto ncreasng or decreasng order. The most common external sortng algorthm stll uses the merge sort as descrbed by Knuth []. In balanced two-way merge, runs (sorted records whch can ft nto the nternal memory) are dstrbuted over two external fles and another two external fles are used to hold the merged runs of the prevous external fles. After each merge the length of runs becomes double. When all the runs are merged nto a sngle run, the process stops. The key drawback of ths process s that t requres extra dsk space. Dufrene and Ln [] and M. R. Islam et al. [3] proposed ther algorthms n whch no external fle s needed; only the orgnal fle s used. M. R. Islam et al. [4][5] proposed an algorthm that uses no extra dsk space and faster than both the algorthms proposed by Dufrene and Ln [] and M. R. Islam et al. [3]. Here we have proposed an effcent external sortng algorthm wth no addtonal dsk space based on the algorthms proposed by Dufrene and Ln [], M. R. Islam et al. [4] and M. R. Islam et al. Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

2 48 [3][6]. The proposed algorthm wll reduce both the tme and I/O complextes of the algorthm proposed by M. R. Islam et al. [4][7].. ALGORITHM REVIEW The proposed algorthm s based on the algorthms proposed by Dufrene and Ln [], M. R. Islam et al. [4] and M. R. Islam et al. [3][7]. Among these algorthms, the overall performance of M. R. Islam et al. [4] s better than others. So, we wll only revew M. R. Islam et al. [4] algorthm n the next subsecton.. A ew External Sortng Algorthm wth o Addtonal Dsk Space wth Specal Inplace Mergng Technque Ths algorthm proposed by M. R. Islam et al. [4], whch works n two phases. In the frst phase the algorthm works same as Dufrene and Ln s [] algorthm wth the only dfference that at the last teraton of frst phase, lower half of man memory s wrtten to the poston of Block_ of the external fle keepng Block_ n the upper half of man memory. In the second phase, the records of Block_S are read nto lower half of man memory. Then the records of lower and upper half of memory are merged usng specal In-place mergng technque whch requres no extra space. After sortng n the memory usng In-place mergng, records of the upper half of memory are wrtten to Block_S and records of Block_S- are read nto upper half of man memory. When the last block Block_3 has been processed, the lower half of man memory contans lowest sorted data among Block_ to Block_S. Then the records of lower half of memory are wrtten n the poston of Block_. Ths process contnues untl Block_S- and Block_S are processed. The I/O complexty of the algorthm s less than that of algorthm proposed by M. R. Islam et al. [3]. 3. A FASTER EXTERAL SORTIG ALGORITHM USIG O ADDITIOAL DISK SPACE The proposed algorthm works n several phases. In the frst phase, the external fle s dvded nto equal szed blocks, where the sze of each block s approxmately equal to the avalable man memory (RAM) of the computer. If the sze of the avalable nternal memory s M then the sze of each block s M and f the sze of external fle s then the number of block, S = /M. Block_ s read nto memory. Then the records of the man memory are sorted usng quck sort and agan wrtten to Block_. The process contnues untl the last block, Block_S, has been processed. After ths the proposed algorthm swtches to ts next phase. Each sorted block s dvded nto two sub-blocks, B_ and B_. The sub block B_ of Block_ and sub block B_ of Block_ are read nto the lower half and upper half of the memory array respectvely. Then the records of the lower half and the upper half of the man Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

3 49 memory, whch are ndvdually sorted are merged usng specal In-place mergng technque [4]. After sortng, the records of the upper half of the man memory are wrtten to B_ of Block_ and the records of the sub block B_ of Block_3 are read nto the upper half of the man memory. Then the records of the man memory are agan merged usng specal In-place mergng technque and the records of the upper half of the man memory are wrtten n the poston of B_ of Block_3 and read the records of sub block B_ of Block_4 to the upper half of the man memory. Repeat ths process untl the records of sub block B_ of Block_S are read nto the upper half of the man memory and processed. ow the lower half of the man memory contans the lowest sorted records among the records Block_ to Block_S and s wrtten n the poston of B_ of Block_. External Fle Block_ Block_ Block_S- Block_S Sorted Records Sorted Records Sorted Records (a) Sorted Records Block_ Block_ Block_S B_ B_ B_ B_ B_ B_ B_ B_ Sub Block (b) Block_ Block_ Block_S B_ B_ B_ B_ B_ B_ Lowest Sorted Records (c) Hghest Sorted Records Block_ Block_ Block_S B_ B_ B_ B_ B_ B_ B_ (d) Fgure. External fle after (a) applyng quck sort (b) splttng each block nto sub blocks, (c) frst teraton (d) renamng the blocks and sub blocks except frst and last sub block. ow sub block B_ of Block_S and sub block B_ of Block_S- are read nto the upper and lower half of the memory array respectvely. Then the records of the lower half and upper half of the man memory are merged usng specal In-place mergng technque. After mergng, the records of the lower half of the man memory are wrtten to B_ of Block_S- and the records of the sub block B_ of Block_S- are read nto the lower half of the man memory. Then the records of the man memory are agan merged usng specal In-place mergng technque and Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

4 50 the records of the lower half of the man memory are wrtten n the poston of B_ of Block_S- and read the records of sub block B_ of Block_S-3 to the lower half of man memory. Repeat ths process untl the records of the sub block B_ of Block_ are read nto the lower half of the man memory and processed. ow the upper half of the man memory contans the hghest sorted records among the records Block_ to Block_S and s wrtten n the poston of B_ of Block_S. At ths pont sub block B_ of Block_ and sub block B_ of Block_S contans the lowest and hghest sorted records respectvely. ow, read B_ of Block_ and B_ of Block_ n the man memory and after mergng, wrte lower and upper half at the poston of B_ of Block_ and B_ of Block_ respectvely. ow rename B_ of Block_ as B_ and B_ of Block_ as B_ and let both B_ and B_ are under Block_. Smlarly merge B_ of Block_ and B_ of Block_3 and after renamng, let they are under Block_. Repeat ths technque untl B_ of Block_S has been processed. Let the last new Block be Block_S. Then apply the above procedure for the new blocks, Block_ to Block_S, to get the next lowest and hghest sorted records. After each teraton the sze of the external fle s decreased by one sub block. Completng all teratons we get the entre fle sorted. Algorthm 3. n Fgure brngs out the whole procedure.. Declare blocks n the external fle to be equal of the avalable man memory. Let the blocks be Block_, Block_,, Block_(S-), Block_S. set P =.. Read Block_P to the man memory. 3. Sort the data of the man memory usng quck sort, wrte the sorted data to Block_P and set P = P +, f P S read Block_P to the man memory and repeat ths step. 4. Dvde each block nto two sub blocks B_ and B_. Here each sub block equals the half of the avalable man memory. 5. Read sub block B_ of Block_ to the lower half of the avalable man memory. Set Q =. 6. Read sub block B_ of Block_Q to the upper half of the avalable man memory. 7. Merge the data of the man memory usng specal In-place mergng [3], wrte upper half to B_ of Block_Q and set Q = Q +, f Q S, read B_ of Block_Q to upper half of the avalable man memory and repeat ths step. 8. Wrte the lower half to B_ of Block_. Fgure. External Sortng Algorthm Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

5 5 9. Read B_ of Block_S to the upper half of the avalable man memory. Set Q = S. 0. Read B_ of Block_Q to the lower half of the avalable man memory.. Merge the data of the man memory usng specal In-place mergng [4], wrte lower half to B_ of Block_Q and set Q = Q. If Q read B_ of Block_Q to lower half of the man memory and repeat ths step.. Wrte the upper half to B_ of Block_S. 3. From B_ of Block_ to B_ of Block_S, declare new blocks n the external fle to be equal of the avalable memory and agan gve the number Block_ to Block_S. Dvde each block nto two sub blocks B_ and B_. Here each sub block equals the half of the avalable man memory. Set P =. 4. Read B_ and B_ of Block_P to the man memory. Merge lower and upper half of Block_P. Wrte the sorted data to Block_P n the external fle and set P = P +. If P S, repeat ths step. 5. If S >, go to step 5. Fgure. External Sortng Algorthm (cont.) 4. COMPLEXITIES AALYSIS OF THE PROPOSED ALGORITHM In ths secton we wll deduce the dsk I/Os and the tme complextes of the proposed algorthm. 4. Input Complexty In the frst phase /M blocks wll have to be processed by quck sort, so t wll take /M read (nput) operatons n the frst phase. In the next phase to obtan the frst lowest sorted sub block t wll take /M read operatons and to obtan the frst hghest sorted sub block t wll also take /M read operatons. ext (/M ) blocks wll have to be processed by specal Inplace mergng to generate the new blocks. So t wll take (/M ) read operatons. Then, to obtan the next lowest and hghest sorted sub blocks t wll take (/M ) read operatons n both cases and so on. So total dsk nput s Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

6 5 ( M + M + M ) + {( M ) + ( M ) + ( M )} { + + } + 3 ( M ) + 3 ( M ) () + 3{ M + ( M ) } + / M + 3 () 4. Output Complexty In the frst phase the proposed algorthm wll take /M wrte (output) operatons. In the next phase to obtan the frst lowest sorted sub block t wll take /M wrte operatons and to obtan the frst hghest sorted sub block t wll take /M wrte operaton. ext (/M ) blocks wll have to be processed to generate the new blocks. So t wll requre (/M ) wrte operatons. Then, to obtan the next lowest and hghest sorted sub block t wll take (/M ) wrte operatons n both cases and so on. So the total output operaton s { M + M + M } + {( M ) + ( M ) + ( M )} { + + } + 3 ( M ) + 3 ( M ) () + 3{ M + ( M ) } + / M + 3 () 4.3 Tme Complexty The tme complexty of the nternal quck sort s O(n log e n) n average case (as gven by Knuth []). Here n s the number of records to be sorted. So, the tme complexty of the frst phase of the proposed algorthm s (/M)(n log e n). In the next phase, the algorthm uses specal In-place mergng technque. The tme complexty of the mergng technque depends on the number of comparson (as gven by Knuth []). To merge n data usng specal In-place mergng technque t wll need n comparsons. So the tme complexty to obtan the frst lowest and hghest sorted sub block s (/M ) n + (/M ) n = (/M ) n. As, after each teraton the fle sze s decreased by one sub block, so the total tme complexty s Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

7 53 {( (n loge n) + ( )n} + [( ) n + {( ) }n] [n + ( ) n] + n M M M M / M / M ( ) (n loge n) + n M + n / M ( ) (n loge n) + 3n M (3) 5. COMPARISO AD DISCUSSIO For smplcty of dscusson we shall use the followng notatons for the algorthms (exstng algorthms) to pont out n ths paper. (a) Dufrene and Ln, An effcent sortng algorthm wth no addtonal dsk space, [], (Algorthm ) (b) M. R. Islam et al., A ew External Sortng Algorthm wth o Addtonal Dsk Space wth Specal In-place Mergng Technque, [4], (Algorthm ) (c) Proposed Algorthm, A Faster External Sortng Algorthm Usng o Addtonal Dsk Space, (Algorthm 3) The I/O and tme complextes of the algorthms proposed by Algorthm, Algorthm and Algorthm 3 are shown n Table. Table : Complextes of the Algorthms proposed by Algorthm, Algorthm and Algorthm 3 Complexty Algorthm Algorthm Algorthm 3 Input Output B B + B + B B B B B + / M + 3 / M + 3 Tme n log n e / B ( ) nlog B e n + n / B / M ( ) (n loge n) +3n M Here the I/O and tme complexty of Algorthm s better than that of Algorthm. So the overall performance of Algorthm s better than Algorthm. So we shall compare and dscuss the complextes of Algorthm and the Algorthm 3. Moreover, the value of M n the Algorthm 3 equals to B n the Algorthm. So, whle performng comparson M has been replaced by B. Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

8 54 5. Comparson of Input Complextes Let, the nput complexty of Algorthm, I = + and B B / B the Algorthm 3, I = + 3 ow, I I = + 3 B B / B = B / B 3 B = Here, for > 6B, (I I ) > 0 or I > I. So, the nput complexty of the Algorthm 3 s less than that of Algorthm for > 6B. 5. Comparson of Output Complextes Let, the output complexty of Algorthm, O = and B B / B the Algorthm 3, O = + 3 ow, O O / B = B 3 B = B / B 3 B Here, for 0B, (O O ) > 0 or O > O. So, the output complexty of the Algorthm 3 s less than that of Algorthm for 0B. Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

9 Comparson of Tme Complextes Let, the tme complexty of Algorthm, T = ( B ) n log e n + n / B and / B the Algorthm 3, T = ( B ) (n log e n) + 3n ow, T T B = ( ) n loge n + n B / / B ( B ) (n log e n) 3n / B = ( ) n log e n + B B n 3n / B / B = ( ) n log e n + n { B B 3 } / B B = ( ) nlog e n B / B / B + n { 3 } Here, for > B, (T T ) > 0 or T > T. So, the tme complexty of the Algorthm 3 s less than that of Algorthm for > B. 6. THEORETICAL RESULTS: The theoretcal results of Algorthm, Algorthm and Algorthm 3 have been shown n tabular format: Table : Reducton of nput Complexty of Algorthm 3 Algorthm and Algorthm External fle sze RAM sze Rato (Algorthm 3 / Algorthm ) Reducton Algorthm Rato (Algorthm 3 / Algorthm ) Reducton Algorthm (%) (%) Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

10 56 Form the Table we see that n case of Algorthm f External Fle 00MB and n case of Algorthm f External Fle > 600MB, reducton of readng nput of Algorthm 3 n percentage s gradually ncreasng. Table 3: Reducton of Output Complexty of Algorthm 3 Algorthm and Algorthm External fle sze RAM sze Rato (Algorthm 3 / Algorthm ) Reducton Algorthm Rato (Algorthm 3 / Algorthm ) Reducton Algorthm (%) (%) Form the Table 3 we see that n case of Algorthm f External Fle 00MB and n case of Algorthm f External Fle 000MB, reducton of wrtng output of Algorthm 3 n percentage s gradually ncreasng. Table 4: Reducton of Tme Complexty of Algorthm 3 Algorthm and Algorthm External fle sze RAM sze Rato (Algorthm 3 / Algorthm ) Reducton Algorthm (%) Rato (Algorthm 3 / Algorthm ) Reducton Algorthm (%) From Table 4 we see that n case of both Algorthm and Algorthm, f External fle sze 00MB, reducton of executon tme of Algorthm 3 n percentage s gradually ncreasng. Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

11 57 7. EXPERIMETAL RESULTS The expermental results of Algorthm, Algorthm and Algorthm 3 have been shown n tabular format below: The confguraton of the system where we have performed the experment: Operatng System : Mcrosoft Wndows XP Professonal, Verson 00, Servce Pack. Processor : Intel (R) Pentum(R) 4 CPU.80GHz Memory : 56MB RD RAM Envronment : Vsual C++.et 003 Table 5: Expermental Results for Input Externa l Fle sze RAM Sze Input for Algorth m () Input for Algorthm () Input for Algorth m (3) Reducton Algorthm (%) Reducton Algorthm (%) Table 6: Expermental Results for Output External Fle sze RAM Sze Output for Algorth m () Output for Algorth m () Output for Algorth m (3) Reducton Algorthm (%) Reducton Algorthm (%) Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

12 58 Table 7: Expermental Results for Executon Tme External Fle sze RAM Sze Tme for Algorthm () n mnutes Tme for Algorthm () n mnutes Tme for Algorthm (3) n mnutes Reducton Algorthm (%) Reducton Algorthm (%) COCLUSIOS We have mproved the performance of the external sortng n the proposed algorthm n terms of both I/O and tme complextes. The algorthm uses quck sort and In-place mergng technque [4] and reduces comparsons to generate mnmum and maxmum sorted sub blocks of records and uses no extra fle. Moreover, the dscusson n the prevous secton, we get the followng results:. The tme complexty of the Algorthm 3 s less than that of Algorthm.. The nput complexty of the Algorthm 3 s less than that of Algorthm for > 6B. 3. The output complexty of Algorthm 3 s less than that of Algorthm for 0B. So, the overall performance of the Algorthm 3 s better than that of Algorthm n terms of both I/O and tme complextes. REFERECES [] D. E. Knuth, Sortng and Searchng, The art of computer programmng Vol. 3, Addson- Wesley, Readng, MA, nd edton, 998. [] W. R. Dufrene, F. C. Ln, An effcent sortng algorthm wth no addtonal space, Compute. J. 35(3) (99). [3] M. R. Islam, S. M. R. Uddn, C. Roy, Computatonal complextes of the external sortng algorthm wth no addtonal dsk space, Internatonal Journal of Computer, Internet and Management (IJCIM), Thaland, Vol. 3, Issue 3, September - December, 005. Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat

13 59 [4] M. R. Islam, W. usrat, M. Hossan, S.M.M. Rana, A ew External Sortng Algorthm wth o Addtonal Dsk Space wth Specal In-place Mergng Technque, Presented at Internatonal Conference on Computer and Informaton Technology (ICCIT), 6-8 December 004, (Dhaka, Bangladesh). [5] M. R. Islam,. Adnan,. Islam, S. Hossen, A new external sortng algorthm wth no addtonal dsk space, Informaton Processng Letters 86(003), [6]. Adnan, R. Islam,. Islam, S. Hossen, A faster hybrd external sortng algorthm wth no addtonal dsk space publshed n the proceedngs of Internatonal Conference on Computer and Informaton Technology (ICCIT), 7-8 December 00 (Dhaka, Bangladesh) [7] S. M. Raqub Uddn, Md. Rafqul Islam, Chnmoy Roy, Complextes of an effcent external sortng algorthm wth specal cases publshed n the proceedngs of Conference on Computer and Informaton Technology (ICCIT), 9- December, 003 (Dhaka, Bangladesh). Jld 8, Bl. (Dsember 006) Jurnal Teknolog Maklumat