Compiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing

Transcription

1 Complng for Parallelsm & Localty Dependence Testng n General Assgnments Deadlne for proect 4 extended to Dec 1 Last tme Data dependences and loops Today Fnsh data dependence analyss for loops General code do 1 = l 1,h 1... do n = l n,h n A(f( 1,..., n )) =... A(g( 1,..., n ))... There exsts a dependence between teratons I=( 1,..., n ) and J=( 1,..., n ) when f(i) = g(j) (l 1,...l n ) < I,J < (h 1,...,h n ) CS553 Lecture Data Dependence Analyss 2 CS553 Lecture Data Dependence Analyss 3 Algorthms for Solvng the Dependence Problem Dependence Testng Heurstcs GCD test (Baneree76,Towle76): determnes whether nteger soluton s possble, no bounds checkng Baneree test (Baneree 79): checks real bounds I-Test (Kong et al. 90): nteger soluton n real bounds Lambda test (L et al. 90): all dmensons smultaneously Delta test (Goff et al. 91): pattern matches for effcency Power test (Wolfe et al. 92): extended GCD and Fourer Motzkn combnaton Use some form of Fourer-Motzkn elmnaton for ntegers, exponental worst-case Parametrc Integer Programmng (Feautrer91) Omega test (Pugh92) Consder the followng code do = 1,5 A(3*+2) = A(2*+1)+1 Queston How do we determne whether one array reference depends on another across teratons of an teraton space? CS553 Lecture Data Dependence Analyss 4 CS553 Lecture Data Dependence Analyss 5 1

2 Dependence Testng: Smple Case Sample code do = l,h A(a*+c 1 ) =... A(a*+c 2 ) do = l,h A(2*+2) = A(2*-2)+1 1 Dependence? a* 1 +c 1 = a* 2 +c 2, or a* 1 a* 2 = c 2 -c 1 Soluton exsts f a dvdes c 2 -c 1 Dependence? 2* 1 2* 2 = -2 2 = -4 (yes, 2 dvdes -4) Knd of dependence? Ant? 2 + d = 1 d = -2 Flow? 1 + d = 2 d = 2 2 CS553 Lecture Data Dependence Analyss 6 CS553 Lecture Data Dependence Analyss 7 GCD Test Generalze test to lnear functons of terators do = l,h do = l,h A(a 1 * + a 2 * + a 0 ) =... A(b 1 * + b 2 * + b 0 )... Agan a 1 * 1 - b 1 * 2 + a 2 * 1 b 2 * 2 = b 0 a 0 Soluton exsts f gcd(a 1,a 2,b 1,b 2 ) dvdes b 0 a 0 do = l,h do = l,h A(4* + 2* + 1) =... A(6* + 2* + 4)... gcd(4,-6,2,-2) = 2 Does 2 dvde 4-1? CS553 Lecture Data Dependence Analyss 8 CS553 Lecture Data Dependence Analyss 9 2

3 Baneree Test Dstance Vectors: Legalty for (=L; <=U; ++) { x[a_0 + a_1*] = = x[b_0 + b_1*] } Does a_0 + a_1* = b_0 + b_1* for some nteger and? If so then (a_1* - b_1* ) = (b_0 - a_0) Determne upper and lower bounds on (a_1* - b_1* ) for (=1; <=5; ++) { x[+5] = x[]; } upper bound = a_1*max() - b_1 * mn( ) = 4 lower bound = a_1*mn() - b_1*max( ) = -4 b_0 - a_0 = Defnton A dependence vector, v, s lexcographcally nonnegatve when the leftmost entry n v s postve or all elements of v are zero Yes: (0,0,0), (0,1), (0,2,-2) No: (-1), (0,-2), (0,-1,1) A dependence vector s legal when t s lexcographcally nonnegatve (assumng that ndces ncrease as we terate) Why are lexcographcally negatve dstance vectors llegal? What are legal drecton vectors? CS553 Lecture Data Dependence Analyss 10 CS553 Lecture Data Dependence Analyss 11 Drecton Vector Defnton A drecton vector serves the same purpose as a dstance vector when less precson s requred or avalable Element of a drecton vector s <, >, or = based on whether the source of the dependence precedes, follows or s n the same teraton as the target n loop do = 1,6 do = 1,5 A(,) = A(-1,-1)+1 Drecton vector: Dstance vector: (<,<) (1,1) CS553 Lecture Data Dependence Analyss 12 Loop-Carred Dependences Defnton A dependence D=(d 1,...d n ) s carred at loop level f d s the frst nonzero element of D do = 1,6 do = 1,6 A(,) = B(-1,)+1 B(,) = A(,-1)*2 Dstance vectors: Loop-carred dependences (0,1) for accesses to A (1,0) for accesses to B The loop carres dependence due to A The loop carres dependence due to B CS553 Lecture Data Dependence Analyss 13 3

4 Parallelzaton Each teraton of a loop may be executed n parallel f t carres no dependences Parallelzaton Each teraton of a loop may be executed n parallel f t carres no dependences do = 1,6 do = 1,5 A(,) = B(-1,-1)+1 B(,) = A(,-1)*2 Parallelze loop? Dstance Vectors: (1,0) for A (flow) (1,1) for B (flow) CS553 Lecture Data Dependence Analyss 14 Iteraton Space do = 1,6 do = 1,5 A(,) = B(-1,-1)+1 B(,) = A(,-1)*2 Parallelze loop? Dstance Vectors: (1,0) for A (flow) (1,1) for B (flow) CS553 Lecture Data Dependence Analyss 15 Iteraton Space 2: Parallelzaton (reprse) Why can t ths loop be parallelzed? 1: Loop Permutaton (reprse) Sample code do = 1,100 A() = A(-1)+1 Why can ths loop be parallelzed? Dstance Vector: (1) do = 1,6 do = 1,5 A(,) = A(,)+1 do = 1,5 do = 1,6 A(,) = A(,)+1 do = 1,100 A() = A() Dstance Vector: (0) Why s ths legal? No loop-carred dependences, so we can arbtrarly change order of teraton executon CS553 Lecture Data Dependence Analyss 16 CS553 Lecture Data Dependence Analyss 17 4

5 Concepts Improve performance by... mprovng data localty parallzng the computaton Next Tme Lecture Loop transformatons for parallelsm and localty Data Dependences teraton space dstance vectors and drecton vectors loop carred Transformaton legalty must respect data dependences scalar expanson as a technque to remove ant and output dependences Data Dependence Testng general formulaton of the problem GCD test CS553 Lecture Data Dependence Analyss 18 CS553 Lecture Data Dependence Analyss 19 5