Optimal Parallel Algorithms for Computing the Sum, the Prefix-sums, and the Summed Area Table on the Memory Machine Models

Transcription

1 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x 1 PAPER Secial Sectio o Secial Sectio o Parallel ad Distributed Comutig ad Netorkig Otimal Parallel Algorithms for Comutig the Sum, the Prefix-sums, ad the Summed Area Table o the Memory Machie Models Koji NAKANO, Member SUMMARY The mai cotributio of this aer is to sho otimal arallel algorithms to comute the sum, the refix-sums, ad the summed area table o to memory machie models, the Discrete Memory Machie (DMM) ad the Uified Memory Machie (UMM). The DMM ad the UMM are theoretical arallel comutig models that cature the essece of the shared memory ad the global memory of GPUs. These models have three arameters, the umber of threads, ad the idth of the memory, ad the memory access latecy l. We first sho that the sum of umbers ca be comuted i O( l l log ) time uits o the DMM ad the UMM. We the go o to sho that Ω( l l log ) time uits are ecessary to comute the sum. We also reset a arallel algorithm that comutes the refix-sums of umbers i O( l l log ) time uits o the DMM ad the UMM. Fially, e sho that the summed area table of size ca be comuted i O( l l log ) time uits o the DMM ad the UMM. Sice the comutatio of the refix-sums ad the summed area table is at least as hard as the sum comutatio, these arallel algorithms are also otimal. key ords: Memory machie models, refix-sums comutatio, arallel algorithm, GPU, CUDA 1. Itroductio The research of arallel algorithms has a log history of more tha 4 years. Sequetial algorithms have bee develoed mostly o the Radom Access Machie (RAM) [1]. I cotrast, sice there are a variety of coectio methods ad atters betee rocessors ad memories, may arallel comutig models have bee reseted ad may arallel algorithmic techiques have bee sho o them. The most ell-studied arallel comutig model is the Parallel Radom Access Machie (PRAM) [] [4], hich cosists of rocessors ad a shared memory. Each rocessor o the PRAM ca access ay address of the shared memory i a time uit. The PRAM is a good arallel comutig model i the sese that arallelism of each roblem ca be revealed by the erformace of arallel algorithms o the PRAM. Hoever, sice the PRAM requires a shared memory that ca be accessed by all rocessors at the same time, it is ot feasible. The GPU (Grahics Processig Uit), is a secialized circuit desiged to accelerate comutatio for buildig ad maiulatig images [5] [8]. Latest GPUs are desiged for geeral urose comutig ad ca erform comutatio i Mauscrit received Jauary 1, 11. Mauscrit revised Jauary 1, 11. The author is ith the Deartmet of Iformatio Egieerig, Hiroshima Uiversity. DOI: /trasif.E.D.1 alicatios traditioally hadled by the CPU. Hece, GPUs have recetly attracted the attetio of may alicatio develoers [5], [9]. NVIDIA rovides a arallel comutig architecture called CUDA (Comute Uified Device Architecture) [1], the comutig egie for NVIDIA GPUs. CUDA gives develoers access to the virtual istructio set ad memory of the arallel comutatioal elemets i NVIDIA GPUs. I may cases, GPUs are more efficiet tha multicore rocessors [11], sice they have hudreds of rocessor cores ad very high memory badidth. CUDA uses to tyes of memories i the NVIDIA GPUs: the global memory ad the shared memory [1]. The global memory is imlemeted as a off-chi DRAM, ad has large caacity, say, Gbytes, but its access latecy is very large. The shared memory is a extremely fast ochi memory ith loer caacity, say, Kbytes. The efficiet usage of the global memory ad the shared memory is a key for CUDA develoers to accelerate alicatios usig GPUs. I articular, e eed to cosider the coalescig of the global memory access ad the bak coflict of the shared memory access [7], [11], [1]. To maximize the badidth betee the GPU ad the DRAM chis, the cosecutive addresses of the global memory must be accessed at the same time. Thus, threads of CUDA should erform coalesced access he they access the global memory. The address sace of the shared memory is maed ito several hysical memory baks. If to or more threads access the same memory baks at the same time, the access requests are rocessed sequetially. Hece to maximize the memory access erformace, threads should access distict memory baks to avoid the bak coflicts of the memory access. I our revious aer [13], e have itroduced to models, the Discrete Memory Machie (DMM) ad the Uified Memory Machie (UMM), hich reflect the essetial features of the shared memory ad the global memory of NVIDIA GPUs. We have reseted that a off-lie ermutatio algorithm develoed for the DMM rus efficietly o a GPU [14]. The outlie of the architectures of the DMM ad the UMM are illustrated i Figure 1. I both architectures, a sea of threads (Ts) are coected to the memory baks (MBs) through the memory maagemet uit (MMU). Each thread is a Radom Access Machie (RAM) [1], hich ca execute fudametal oeratios i a time uit. We do ot discuss the architecture of the sea of threads i this aer, but e ca imagie that it cosists of a set of multi-core ro- Coyright c x The Istitute of Electroics, Iformatio ad Commuicatio Egieers

2 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x cessors hich ca execute may threads i arallel. Threads are executed i SIMD [15] fashio, ad the rocessors ru o the same rogram ad ork o the differet data. The DMM ad the UMM have three arameters: idth, latecy l, ad the umber of threads. The idth is the umber of MBs. Also, they ca disatch threads i threads ad the threads ca sed memory access requests. The latecy l is the umber of clock cycles to comlete the memory access requests. MBs costitute a sigle address sace of the memory. A sigle address sace of the memory is maed to the MBs i a iterleaved ay such that the ord of data of address i is stored i the (i mod )-th bak, here is the umber of MBs. We ca thik that addresses j, j 1,..., (1) j 1 for each j costitute a address grou. The mai differece of the to architectures is the coectio of the address lie betee the MMU ad the MBs, hich ca trasfer a address value. I the DMM, the address lies coect the MBs ad the MMU searately, hile a sigle set of address lies from the MMU is coected to the MBs i the UMM. Hece, i the UMM, the same address value is broadcast to every MB, ad the same address of the MBs ca be accessed i each time uit. O the other had, differet addresses of the MBs ca be accessed i the DMM. Hece, the DMM allos to access ay address grou of each MB at the same time, hile the UMM oly allos to access the same address grou of all MBs. Sice the memory access of the UMM is more restricted tha that of the DMM, the UMM is less oerful tha the DMM. a sea of threads a sea of threads ruig o multicore rocessors ruig o multicore rocessors MMU MB MB MB MB Fig. 1 DMM address lie MMU MB MB MB MB UMM data lie The architectures of the DMM ad the UMM The erformace of algorithms of the PRAM is usually evaluated usig to arameters: the size of the iut ad the umber of rocessors. For examle, it is ell ko that the sum of umbers ca be comuted i O( log ) time o the PRAM []. We use four arameters, the size of the iut, the umber of threads, the idth ad the latecy l of the memory he e evaluate the erformace of algorithms o the DMM ad o the UMM. The idth is the umber of memory baks ad the latecy l is the umber of time uits to comlete the memory access. Hece, the erformace of algorithms o the DMM ad the UMM is evaluated as a fuctio of (the size of a roblem), (the umber of threads), (the idth of a memory), ad l (the latecy of a memory). I NVIDIA GPUs, the idth of global ad shared memory is 16 or 3. Also, the latecy l of the global memory is several hudreds clock cycles. A grid ca have at most blocks ith at most 51 threads each for CUDA Versio 1.x ad 31 1 blocks ith at most 14 threads each for CUDA Versio 3. or later [1]. Thus, the umber of threads ca be 33 millio for CUDA Versio 1.x ad much more for CUDA Versio 3. or later. Suose that a array a of umbers is give. The refix-sums of a is the array of size such that the i-th ( i 1) elemet is a[] a[1] a[i]. I this aer, e cosider the additio as a biary oerator for the refix-sums, but it ca be geeralized for ay associative biary oerator. Clearly, a sequetial algorithm ca comute the refix-sums by executig a[i 1] a[i 1] a[i] for all i ( i 1) i tur. The comutatio of the refix-sums of a array is oe of the most imortat algorithmic rocedures. May algorithms such as grah algorithms, geometric algorithms, image rocessig ad matrix comutatio call refix-sum algorithms as a subroutie. For examle, the refix-sums comutatio is used to obtai the re-order, the i-order, ad the ost-order of a rooted biary tree i arallel []. So, it is very imortat to develo efficiet arallel algorithms for the refix-sums. Suose that a matrix a of size is give. Let a[i][ j]( i, j 1) deote the elemet at the i-th ro ad j-th colum. The summed area table [16] is a matrix b of the same size such that b[i][ j] = i i, j j a[i ][ j ]. It should have o difficulty to cofirm that the summed area table ca be obtaied by comutig the colum-ise refixsums of the ro-ise refix-sums as illustrated i Figure. Alteratively, e ca comute the ro-ise refix-sums of the colum-ise refix-sums to obtai the same matrix. Oce e have the summed area table, the sum of ay rectagular area of a ca be comuted i O(1) time. More secifically, e have the folloig equatio: a[i][ j] = b[d][r] b[u][l] b[d][l] b[u][r]. u<i d,l< j r Thus, the sum of a rectagular area ca be comuted usig four elemets of the summed area table b. Sice the sum of a rectagular area ca be comuted i O(1) time the summed area table has may alicatios i the are of image rocessig [17]. The mai cotributio of this aer is to sho otimal arallel algorithms comutig the sum ad the refixsums o the DMM ad the UMM. We first sho that the sum of umbers ca be comuted i O( l l log ) time uits usig threads o the DMM ad the UMM ith idth ad latecy l. We the go o to discuss the loer boud of the time comlexity ad sho three loer bouds, Ω( l )-time badidth limitatio, Ω( )-time latecy

3 NAKANO: OPTIMAL PARALLEL ALGORITHMS FOR COMPUTING THE SUM, THE PREFIX-SUMS, AND THE SUMMED AREA TABLE ON THE MEMORY MACHINE MODELS ro-ise refix-sums colum-ise refix-sums Fig. Summed area table limitatio, ad Ω(l log )-time reductio limitatio. From this discussio, the comutatio of the sum takes at least Ω( l l log ) time uits o the DMM ad the UMM. Thus, the sum algorithm is otimal. For the comutatio of the refix-sums, e first evaluate the comutig time of a ell-ko aive algorithm [4], [17]. We sho that a aive refix-sums algorithm rus i O( log l log l log ) time. Hece, this aive algorithm is ot otimal ad it has a overhead of factor log both for the badidth limitatio ad for the latecy limitatio l. Next, e sho a otimal arallel algorithm that comutes the refix-sums of umbers i O( l l log ) time uits o the DMM ad the UMM. Hoever, this algorithm uses ork sace of size ad it may ot be accetable if the size of the iut is very large. We also sho that the refix-sums ca also be comuted i the same time uits, eve if ork sace ca store oly mi( log,l log(l)) umbers. Fially, e sho that the summed area table ca be comuted i O( l l log ) time uits o the DMM ad the UMM. Several techiques for comutig the refix-sums o GPUs have bee sho i [17]. They have reseted a comlicated data routig techique to avoid the bak coflict i the comutatio of the refix-sums. Hoever, their algorithm erforms memory access distat locatios i arallel ad it erforms o-coalesced memory access. Hece it is ot efficiet for the UMM, that is, the global memory of GPUs. I [17] a ork-efficiet arallel algorithm for refixsums o the GPU has bee reseted. Hoever, the algorithm uses ork sace of log, ad also the theoretical aalysis of the erformace has ot bee reseted. This aer is orgaized as follos. Sectio briefly defies the Discrete Memory Machie (DMM) ad the Uified Memory Machie (UMM) itroduced i our revious aer [13]. I Sectio 3, e evaluate the comutig time of the cotiguous memory access the memory of the DMM ad the UMM. The cotiguous memory access is a key igrediet of arallel algorithm develomet o the DMM ad the UMM. Usig the cotiguous access, e sho that the sum of umbers ca be comuted i O( l l log ) time uits i Sectio 4. We the go o to discuss the loer boud of the time comlexity ad sho three loer bouds, Ω( l )-time badidth limitatio, Ω( )-time latecy limitatio, ad Ω(l log )-time reductio limitatio i Sectio 5. Sectio 6 shos a aive algorithm for the refixsums, hich rus i O( log l log l log ) time uits. We the reset a otimal arallel algorithm for the refixsums ruig i O( l l log ) time uits i Sectio 7. Fially, e sho that the summed area table ca be comuted i O( l l log ) time uits i Sectio 8. Sectio 9 offers coclusio of this aer.. Parallel Memory Machies: DMM ad UMM The mai urose of this sectio is to defie the Discrete Memory Machie (DMM) ad the Uified Memory Machie (UMM) itroduced i our revious aer [13]. Please see [13] for the details. We first defie the Discrete Memory Machie (DMM) of idth ad latecy l. Let m[i] (i ) deote a memory cell of address i i the memory. Let B[ j] = {m[ j], m[ j ], m[ j ], m[ j 3],...} ( j 1) deote the j-th bak of the memory. Clearly, a memory cell m[i] is i the (i mod )-th memory bak. We assume that memory cells i differet baks ca be accessed i a time uit, but o to memory cells i the same bak ca be accessed i a time uit. Also, e assume that l time uits are ecessary to comlete a access request ad cotiuous requests are rocessed i a ielie fashio through the MMU. Thus, if oly oe request is set to a bak, l time uits are ecessary to comlete it. I geeral, it takes k l 1 time uits to comlete k ( 1) access requests to a articular bak. We assume that threads o the DMM ad the UMM ith idth are artitioed ito grous of threads called ars. It makes sese to assume that ad is a multile of. More secifically, threads are artitioed ito ars W(), W(1),..., W( 1) such that W(i) = {T(i ), T(i 1),...,T((i 1) 1)} ( i 1). I other ords, the umber of MBs ad the umber of threads i a ar are equal. Wars are activated for memory access i tur, ad threads i a ar try to access the memory at the same time. I other ords, W(), W(1),...,W( 1) are activated i a roud-robi maer if at least oe thread i a ar requests memory access. If o thread i a ar eeds memory access, such ar is ot activated for memory access. Whe W(i) is activated, threads i W(i) seds memory access requests, oe request er thread, to the memory. We also assume that a thread caot sed a e memory access request util the revious memory access request is comleted. Hece, if a thread sed a memory access request, it must ait l time uits to sed a e memory access request. We ext defie the Uified Memory Machie (UMM) of idth as follos. Let A[ j] = {m[ j ], m[ j

4 4 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x 1],...,m[(j 1) 1]} deote the j-th address grou. We assume that memory cells i the same address grou are rocessed at the same time. Hoever, if they are i the differet grous, oe time uit is ecessary for each of the grous. Also, similarly to the DMM, threads are artitioed ito ars ad each ar accesses the memory i tur. Figure 3 shos examles of memory access o the DMM ad the UMM. We ca thik that the memory access is erformed through l-stage ielie registers as illustrated i the figure. We assume that each memory access request is comleted he it reaches the last ielie stage. To ars W() ad W(1) access to m[7], m[5], m[15], m[] ad m[1], m[11], m[1], m[9], resectively. I the DMM, memory access requests by W() are searated ito to ielie stages, because addresses m[7] ad m[15] are i the same bak B(3). Those by W(1) occuies 1 stage, because all requests are i distict baks. Thus, the memory requests occuy three stages, it takes = 7 time uits to comlete the memory access. I the UMM, memory access requests by W() are destied for three address grous. Hece the memory requests occuy three stages. Similarly those by W(1) occuy to stages. Hece, it takes = 9 time uits to comlete the memory access. Throughout of this aer, ithout losig geerality, e assume that the idth of the DMM ad the UMM, the umber of the threads, ad the size of roblem iut are oers of to. The reader should have o difficulty to modify algorithms reseted i this aer to ork correctly eve if some of these values are ot oers of to usig stadard algorithmic techiques. 3. Cotiguous Memory Access The mai urose of this sectio is to revie the cotiguous memory access o the DMM ad the UMM sho i [13]. Suose that a array a of size ( ) is give. We use threads to access all of elemets i a such that each thread accesses elemets. Note that accessig ca be readig from or ritig i. Let a[i] ( i 1) deote the i-th elemet i a. Whe, the cotiguous access ca be erformed as follos: [Cotiguous memory access] for t to 1do for i to 1 do i arallel T(i) access a[ t i] We ill evaluate the comutig time. For each t ( t 1), threads access elemets a[t], a[t 1],...,a[(t 1) 1]. This memory access is erformed by ars i tur. More secifically, first, threads i W() access a[t], a[t1],..., a[t 1]. After that, threads i W(1) access a[t ], a[t 1],...,a[t 1], ad the same oeratio is reeatedly erformed. I geeral, threads i W( j)( j 1) accesses a[t j], a[t j 1],...,a[t ( j 1) 1]. Sice elemets are accessed by a ar are i the differet bak, the access ca be comleted i l time uits o the DMM. Also, these elemets are i the same address grou, ad thus, the access ca be comleted i l time uits o the UMM. Recall that the memory access are rocessed i ielie fashio such that threads i each W( j) sed memory access requests i oe time uit. Hece, threads i ars sed memory access requests i time uits. After that, the last memory access requests by W( 1) are comleted i l 1 time uits. Thus, threads access elemets a[t], a[t 1],...,a[(t 1) 1] i l 1 time uits. Sice this memory access is reeated times, the cotiguous access ca be doe i ( l 1) = O( l ) time uits. If < the, the cotiguous memory access ca be simly doe usig threads out of the threads. If this is the case, the memory access ca be doe by O( l) time uits. Therefore, e have, Lemma 1: The cotiguous access to a array of size ca be doe i O( l l) time uits usig threads o the UMM ad the DMM ith idth ad latecy l. We ca cosider that memory access is cotiguous if threads i every ar access the cotiguous addresses of the shared memory o the DMM or the global memory of the UMM. It should have o difficulty to cofirm that, if all of the threads erforms such cotiguous memory access to umbers, umbers er thread, the memory access is also comleted i O( l l) time uits. 4. A otimal arallel algorithm for comutig the sum The mai urose of this sectio is to sho a otimal arallel algorithm for comutig the sum o the memory machie models. Let a be a array of = m umbers. Let us sho a algorithm to comute the sum a[]a[1] a[ 1]. The algorithm uses a ell-ko arallel comutig techique hich reeatedly comutes the airise sums. We imlemet this techique to erform cotiguous memory access. The details are selled out as follos: [Otimal algorithm for comutig the sum] for t m 1 doto do for i to t 1 do i arallel a[i] a[i] a[i t ] Figure 4 illustrates ho the sums of airs are comuted. From the figure, the reader should have o difficulty to cofirm that this algorithm comute the sum correctly. We assume that threads are used to comute the sum. For each t ( t m 1), t oeratios a[i] a[i] a[i t ] are erformed. These oeratio ivolve the folloig memory access oeratios: readig from a[], a[1],...,a[ t 1], readig from a[ t ], a[ t 1],...,a[ t 1], ad ritig i a[], a[1],...,a[ t 1].

5 NAKANO: OPTIMAL PARALLEL ALGORITHMS FOR COMPUTING THE SUM, THE PREFIX-SUMS, AND THE SUMMED AREA TABLE ON THE MEMORY MACHINE MODELS 5 l-stage ielie registers DMM W() W(1) B[] B[1] B[] B[3] Each ielie stage stores memory access requests destied for the differet baks UMM W() W(1) l-stage ielie registers A[] A[1] A[] A[3] Each ielie stage stores memory access requests destied for the same address grou Fig. 3 Examles of memory access o the DMM ad the UMM Fig. 4 Illustratig the summig algorithm for umbers Sice these memory access oeratios are cotiguous, they ca be doe i O( t t l l) time usig threads both o the DMM ad o the UMM ith idth ad latecy l from Lemma 1. Thus, the total comutig time is t= m 1 O( t t l l) = O( m m l lm) = O( l l log ) ad e have, Lemma : The sum of umbers ca be comuted i O( l l log ) time uits usig threads o the DMM ad o the UMM ith idth ad latecy l. 5. The loer boud of the comutig time ad the latecy hidig Let us discuss the loer boud of the time ecessary to comute the sum o the DMM ad the UMM to sho that our arallel summig algorithm for Lemma is otimal. Sice the sum is the last value of the refix-sums ad the summed area table, this loer boud discussio for the sum ca be alied to that for the refix-sums ad the summed area table. We ill sho three loer bouds of the sum, Ω( l )-time badidth limitatio, Ω( )-time latecy limitatio, ad Ω(l log )-time reductio limitatio. Sice the idth of the memory is, at most umbers i the memory ca be read i a time uit. Clearly, all of the umbers must be read to comute the sum. Hece, Ω( ) time uits are ecessary to comute the sum. We call the Ω( )-time loer boud the badidth limitatio. Sice the memory access takes latecy l, a thread ca sed at most t l memory read requests i t time uits. Thus, threads ca sed at most t l total memory requests i t time uits. Sice at least umbers i the memory must be read to comute the sum, t l must be satisfied. Thus, at least t =Ω( l l ) time uits are ecessary. We call the Ω( )-time loer boud the latecy limitatio. Next, e ill sho the reductio limitatio, the Ω(l log )-time loer boud. The formal roof is more comlicated tha those for the badidth limitatio ad the latecy limitatio. Imagie that each of iut umbers stored i the shared memory (or the global memory) is a toke ad each thread is a box. Wheever to tokes are laced i a box,

6 6 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x they are merged ito oe immediately. We ca move tokes to boxes ad each box ca accet at most oe toke i l time uits. Suose that e have tokes outside boxes. We ill rove that it takes at least l log time uits to merge them ito oe toke. For this urose, e ill rove that, if e have tokes at some time, e must have at least tokes l time uits later. Suose that e have tokes such that k of them are i k boxes ad the remaiig k tokes are out of boxes. If k k the e ca move k tokes to k boxes ad ca merge k airs of tokes i l time uits. After that, k tokes remai. If k > k the e ca merge k airs of tokes ad e have k tokes after l time uits. Hece, after l time uits, e have at least max( k, k) tokes. Thus, e must have at least tokes l time uits later. I other ords, it is ot ossible to reduce the umber of tokes by half or less. Thus, it takes at least l log time uits to merge tokes ito oe. It should be clear that, readig a umber by a thread from the shared memory (or the global memory) corresods to a toke movemet to a box. Therefore, it takes at least Ω(l log ) time uits to comute the sum of umbers. From the discussio above, e have, Theorem 3: Both the DMM ad the UMM ith threads, idth, ad latecy l take at least Ω( l l log ) time uits to comute the sum of umbers. From Theorem 3, the arallel algorithm for commutig the sum sho for Lemma is otimal. Let us discuss the relatio of three limitatios. From a ractical oit of vie, idth ad latecy l are costat values that caot be chaged by arallel comuter users. These values are fixed he a arallel comuter based o the memory machie models is maufactured. Also, the size of the iut are variable. Programmers ca adjust the umber of threads to obtai the best erformace. Thus, the value of the latecy limitatio l ca be chaged by rogrammers. Let us comare the values of three limitatios. l : From l, the badidth limitatio domiates the latecy limitatio. l log : From l log, the badidth limitatio domiates the reductio limitatio. l : From log l log, the latecy limitatio domiates the reductio limitatio. Thus, if both l ad l log are satisfied, the comutig time of the sum algorithm for Lemma is O( ). As illustrated i Figure 3, the memory machie models have l-stage ielie registers such that each stage has registers. Sice more tha oe memory requests by a thread ca ot be stored i each ielie register, l must be satisfied to fill all the ielie registers ith memory access requests by threads. Sice the sum algorithm has log log stages ad exected memory access requests are set to the ielie registers, l log must also be satisfied to fill all the ielie registers ith log memory access requests. From the discussio above, to hide the latecy, the umber of threads must be at least the umber l of ielie registers ad the size of iut must be at least l log(l). 6. A aive refix-sums algorithm We assume that a array a ith = m umbers is give. Let us start ith a ell-ko aive refix-sums algorithm for array a [17], [18], ad sho it is ot otimal. The aive refix-sums algorithm is ritte as follos: [A aive refix-sums algorithm] for t tom 1do for i t to 1 do i arallel a[i] a[i] a[i t ] Figure 5 illustrates ho the refix-sums are comuted. We assume that threads are available ad evaluate the comutig time of the aive refix-sums algorithm. The folloig three memory access oeratios are erformed for each t ( t m 1) by: readig from a[], a[1],...,a[ t 1], readig from a[ t ], a[ t 1],...,a[ 1], ad ritig i a[ t ], a[ t 1],...,a[ 1]. Each of the three oeratios ca be doe by cotiguous memory access for t elemets. Hece, the comutig time of each t is O( t ( t )l l) from Lemma 1. The total comutig time is: m 1 O( t ( t )l l) t= = O( log Thus, e have, l log l log ). Lemma 4: The aive refix-sum algorithm rus i O( log l log l log ) time uits usig threads o the DMM ad o the UMM ith idth ad latecy l. If the comutig time of Lemma 4 matches the loer boud sho i Theorem 3, the refix-sum algorithm is otimal. Hoever, it does ot match the loer boud. I the folloig sectio, e ill sho a otimal refix-sum algorithm hose ruig time matches the loer boud. 7. Otimal refix-sum algorithm This sectio shos a otimal algorithm for the refix-sums ruig i O( l l log ) time uits. We use m arrays a, a 1,...a m 1 as ork sace. Each a t ( t m 1) ca store t umbers. Thus, the total size of the m arrays is o more tha 1 m 1 1 = m 1 = 1. We assume that the iut of umbers are stored i array a m of size. The algorithm has to stages. I the first stage, iterval sums are stored i the m arrays. The secod stage uses iterval sums i the m arrays to comute the resultig

7 NAKANO: OPTIMAL PARALLEL ALGORITHMS FOR COMPUTING THE SUM, THE PREFIX-SUMS, AND THE SUMMED AREA TABLE ON THE MEMORY MACHINE MODELS Fig. 5 Illustratig the aive refix-sums algorithm for umbers refix-sums. The details of the first stage are selled out as follos. [Comute the iterval sums] for t m 1 doto do for i to t 1 do i arallel a t [i] a t1 [ i] a t1 [ i 1] Figure 6 illustrates ho the iterval sums are comuted. Whe this rogram termiates, each a t [i]( t m 1, i t 1) stores a t [i ]a t t [i 1] a t t [(i1) 1]. t I the secod stage, the refix-sums are comuted by comutig the sums of the iterval sums as follos: [Comute the sums of the iterval sums] for t tom 1do for i to t 1 do i arallel a t1 [ i 1] a t [i] for i to t do i arallel a t1 [ i ] a t1 [ i ] a t [i] Figure 7 shos ho the refix-sums are comuted. I the figure, a t1 [ i 1] a t [i] ad a t1 [ i ] a t1 [ i ] a t [i] corresod to coy ad add, resectively. Whe this algorithm termiates, each a m [i] ( i t 1) stores the refix-sum a m [] a m [1] a m [i]. We assume that threads are available ad evaluate the comutig time. The first stage ivolves the folloig memory access oeratios for each t ( t m 1): readig from a t1 [], a t1 [],...,a t1 [ t1 ], readig from a t1 [1], a t1 [3],...,a t1 [ t1 1], ad ritig i a t [], a t [1],...,a t [ t 1]. Every to addresses is accessed i the readig oeratios. Thus, these three memory access oeratios are essetially cotiguous access ad they ca be doe i O( t l) time uits. Therefore, the total comutig time of the first stage is t=1 t l m 1 O( t t l l) = O( l l log ). The secod stage cosists of the folloig memory access oeratios for each t ( t m 1): readig from a t [], a t [1],...,a t [ t 1], readig from a t1 [], a t1 [4],...,a t1 [ t1 ], ad ritig i a t1 [], a t1 [1],...,a t1 [ t1 1]. Similarly, these oeratios ca be doe i O( t t l l) time uits. Hece, the total comutig time of the secod stage is also O( l l log ). Thus, e have, Theorem 5: The refix-sums of umbers ca be comuted i O( l l log ) time uits usig threads o the DMM ad o the UMM ith idth ad latecy l if ork sace of size is available. From Theorem 3, the loer boud of the comutig time of the refix-sums is Ω( l l log ). Suose that is very large ad ork sace of size is ot available. We ill sho that, if ork sace o smaller tha mi( log,l log(l)) is available, the refixsums ca also be comuted i O( l l log ). Let k be a arbitrary umber such that k. We artitio the iut a ith umbers ito k grous ith k ( ) umbers each. Each t-th grou ( t k 1) has k umbers a[tk], a[tk 1],...,a[(t 1)k 1]. The refix-sums of every grou is comuted usig threads i tur as follos. [Sequetial-arallel refix-sums algorithm] for t to k 1do begi if(t ) a[tk] a[tk] a[tk 1] comute the refix-sums of k umbers a[tk], a[tk 1],...,a[(t 1)k 1] ed It should be clear that this algorithm comutes the refixsums correctly. The refix-sums of k umbers ca be comuted i O( k kl l log k). Sice the comutatio of the refix-sums is reeated k times, the total comutig time is O( k kl l log k) k = O( l l log k k ). Thus, e have, Corollary 6: The refix-sums of umbers ca be comuted i O( l l log k k ) time uits usig threads o the DMM ad o the UMM ith idth ad latecy l if ork sace of size k is available.

8 8 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x a a a a a -15 Fig. 6 Illustratig the comutatio of iterval sums i m arrays. a 4 coy add a a a a -15 Fig. 7 Illustratig the comutatio of the sums of the iterval sums i m arrays. l log k l log( log ) k log l log(l log(l)) l log(l) If k log the, = O( l ). If k l log k l log(l) the k = O( ). Thus, if k mi( log,l log(l)) the the comutig time is O( l ). 8. Otimal algorithms for the summed area table The mai urose of this sectio is to sho otimal algorithms for comutig the summed area table. It should be clear that the colum-ise refix-sums ca be obtaied by the trasose, the ro-ise refix-sums comutatio, ad the trasose. I our revious aer [13], e have reseted that the trasose of a matrix of size ca be doe i O( l ) time uits both o the DMM ad the UMM ith idth ad latecy l usig threads. As illustrated i Figure the summed area table ca be comuted by the roise refix-sums ad the colum-ise refix-sums. Thus, if the ro-ise refix-sums of a matrix of size ca be comuted i O( l l log ) time uits, the summed area table ca be comuted i the same time uits. For simlicity, e assume that is a oer of to. Recall that the algorithm for Theorem 5 comutes the refix-sums of umbers. Imagie that the this algorithm is executed o a matrix of size such that umbers i a matrix arraged i a 1-dimesioal array i ro-major order. We ill sho that the ro-ise refix-sums ca be comuted by modifyig the algorithm for Theorem 5, hich has folloig to stages: Stage 1 the iterval sums are comuted as illustrated i Figure 6, ad Stage the sums of the iterval sums are comuted as illustrated i Figure 7. The idea is to omit several oeratios erformed i these to stages such that the iterval sums of a ro is ot roagated to the ext ro. We ca thik that a t ( log t log ) i these stages is a matrix of size t such that a t [i t j] ( i 1, j t 1) is a elemet i the i-th ro ad j-th colum. For examle, the sizes of a 4, a 3, ad a i Figures 6 ad 7 are 4 4, 4 ad 4 1, resectively. Hece, a 4 [], a 4 [1], a 4 [], a 4 [3], a 3 [], a 3 [1], ad a [] are i ro. We modify Stage 1 such that the iterval sums ithi each ro is comuted. For this urose, e execute the first log (= m ) stes of Stage 1 ad the remaiig m stes are omitted as follos: [Comute the iterval sums i ro-ise] for t m 1 doto m do for i to t 1 do i arallel a t [i] a t1 [ i] a t1 [ i 1] For examle, i Figure 6, a 3 ad a are comuted, but the comutatio of a 1 ad a is omitted.

9 NAKANO: OPTIMAL PARALLEL ALGORITHMS FOR COMPUTING THE SUM, THE PREFIX-SUMS, AND THE SUMMED AREA TABLE ON THE MEMORY MACHINE MODELS 9 coy add a a a ro ro 1 ro ro 3 Fig. 8 Illustratig the comutatio of the sums of the iterval sums of every ro Next, e modify Stage such that the comutatio m is erformed ithi each ro. First, e omit the first stes ad execute the remaiig m stes. I the remaiig m stes, oeratios from a ro to the ext ro are omitted. The reader should comare Figure 7 ad Figure 8 to see ho Stage is modified. Note that all coy oeratios are alays erformed ithi the same ro. Hoever, add oeratios may be erformed to ext ros. For examle, add oeratio a 3 [] a 3 [] a [] is omitted because a 3 [] is i ro 1 ad a [] is i ro. The details are selled out as follos: [Comute the sums of the iterval sums i ro-ise] for t m to m 1do for i to t 1 do i arallel a t1 [ i 1] a t [i] for i to t do i arallel if a t1 [ i ] ad a t [i] are i the same ro the a t1 [ i ] a t1 [ i ] a t [i] Clearly the comutatio of modified Stages 1 ad does ot exceed their origial stages sho for Theorem 5. Thus, e have, Theorem 7: The summed area table of a matrix of size ca be comuted i O( l l log ) time uits usig threads o the DMM ad o the UMM ith idth ad latecy l, if ork sace of size is available. From Theorem 3, the arallel algorithm for Theorem 7 is otimal. 9. Coclusio The mai cotributio of this aer is to sho arallel algorithms to comute the sum, the refix-sums, ad the summed area table i O( l l log ) time uits usig threads o the DMM ad the UMM ith idth ad latecy l. We have also sho ay algorithm to comute these values takes Ω( l l log ) time uits. We believe that to memory machie models, the DMM ad the UMM are romisig as latforms of develomet of algorithmic techiques for GPUs. We la to develo efficiet algorithms for grah-theoretic roblems, geometric roblems, ad image rocessig roblems o the DMM ad the UMM. Refereces [1] A.V. Aho, J.D. Ullma, ad J.E. Hocroft, Data Structures ad Algorithms, Addiso Wesley, [] A. Gibbos ad W. Rytter, Efficiet Parallel Algorithms, Cambridge Uiversity Press, [3] A. Grama, G. Karyis, V. Kumar, ad A. Guta, Itroductio to Parallel Comutig, Addiso Wesley, 3. [4] M.J. Qui, Parallel Comutig: Theory ad Practice, McGra- Hill, [5] W.W. Hu, GPU Comutig Gems Emerald Editio, Morga Kaufma, 11. [6] Y. Ito, K. Ogaa, ad K. Nakao, Fast ellise detectio algorithm usig Hough trasform o the GPU, Proc. of Iteratioal Coferece o Netorkig ad Comutig, , Dec. 11. [7] D. Ma, K. Uda, Y. Ito, ad K. Nakao, A GPU imlemetatio of comutig euclidea distace ma ith efficiet memory access, Proc. of Iteratioal Coferece o Netorkig ad Comutig,.68 76, Dec. 11. [8] A. Uchida, Y. Ito, ad K. Nakao, Fast ad accurate temlate matchig usig ixel rearragemet o the GPU, Proc. of Iteratioal Coferece o Netorkig ad Comutig, , Dec. 11. [9] K. Nishida, Y. Ito, ad K. Nakao, Acceleratig the dyamic rogrammig for the matrix chai roduct o the GPU, Proc. of Iteratioal Coferece o Netorkig ad Comutig,.3 36, Dec. 11. [1] NVIDIA Cororatio, NVIDIA CUDA C rogrammig guide versio 4., 1. [11] D. Ma, K. Uda, H. Ueyama, Y. Ito, ad K. Nakao, Imlemetatios of a arallel algorithm for comutig euclidea distace ma i multicore rocessors ad GPUs, Iteratioal Joural of Netorkig ad Comutig, vol.1,.6 76, July 11. [1] NVIDIA Cororatio, NVIDIA CUDA C best ractice guide versio 3.1, 1. [13] K. Nakao, Simle memory machie models for GPUs, Proc. of Iteratioal Parallel ad Distributed Processig Symosium Workshos, , May 1. [14] A. Kasagi, K. Nakao, ad Y. Ito, A imlemetatio of coflictfree off-lie ermutatio o the GPU, to aear i Proc. of Iteratioal Coferece o Netorkig ad Comutig,.6 3, 1. [15] M.J. Fly, Some comuter orgaizatios ad their effectiveess, IEEE Trasactios o Comuters, vol.c-1, , 197. [16] F. Cro, Summed-area tables for texture maig, Proc. of the 11th aual coferece o Comuter grahics ad iteractive techiques,.7 1, [17] M. Harris, S. Seguta, ad J.D. Oes, Chater 39. arallel refix sum (sca) ith CUDA, i GPU Gems 3, Addiso-Wesley, 7. [18] W.D. Hillis ad G.L. Steele, Jr., Data arallel algorithms, Commu. ACM, vol.9, o.1, , Dec

10 1 IEICE TRANS. INF. & SYST., VOL.Exx??, NO.xx XXXX x Koji Nakao received the BE, ME ad Ph.D degrees from Deartmet of Comuter Sciece, Osaka Uiversity, Jaa i 1987, 1989, ad 199 resectively. I , he as a Research Scietist at Advaced Research Laboratory. Hitachi Ltd. I 1995, he joied Deartmet of Electrical ad Comuter Egieerig, Nagoya Istitute of Techology. I 1, he moved to School of Iformatio Sciece, Jaa Advaced Istitute of Sciece ad Techology, here he as a associate rofessor. He has bee a full rofessor at School of Egieerig, Hiroshima Uiversity from 3. He has ublished extesively i jourals, coferece roceedigs, ad book chaters. He served o the editorial board of jourals icludig IEEE Trasactios o Parallel ad Distributed Systems, IEICE Trasactios o Iformatio ad Systems, ad Iteratioal Joural of Foudatios o Comuter Sciece. He has also guest-edited several secial issues icludig IEEE TPDS Secial issue o Wireless Netorks ad Mobile Comutig, IJFCS secial issue o Grah Algorithms ad Alicatios, ad IEICE Trasactios secial issue o Foudatios of Comuter Sciece. He has orgaized cofereces ad orkshos icludig Iteratioal Coferece o Netorkig ad Comutig, Iteratioal Coferece o Parallel ad Distributed Comutig, Alicatios ad Techologies, IPDPS Worksho o Advaces i Parallel ad Distributed Comutatioal Models, ad ICPP Worksho o Wireless Netorks ad Mobile Comutig. His research iterests icludes image rocessig, hardare algorithms, GPUbased comutig, FPGA-based recofigurable comutig, arallel comutig, algorithms ad architectures.