Compilation Techniques for Embedded Data Parallel Languages

Size: px

Start display at page:

Download "Compilation Techniques for Embedded Data Parallel Languages"

Shona Reed
7 years ago
Views:

1 Compilation Techniques for Embedded Data Parallel Languages Bryan Catanzaro Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS May 11, 2011

2 Copyright 2011, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Acknowledgement This research has been supported by a National Science Foundation Graduate Research Fellowship, an NVIDIA Graduate Fellowship, a Qualcomm Innovation Fellowship, the Gigascale Research Center, and the Parallel Computing Laboratory, which itself is supported by Microsoft (Award #024263), Intel (Award #024894), and by matching funding from U.C. Discovery (Award #DIG ), with additional support from affiliates National Instruments, NEC, Nokia, NVIDIA, and Samsung.

11 Ax A x

12 ψ = 1 ψ = 10 ψ = 100 ψ = 1000 ψ = ψ =

14 P

22 3 5 9

29 1/5 1/3

35 E x E 1,...,E n A E E 1 + E 2 E 1 < E 2 E... E 1 E 2 E 1 E 2 E 1 E p E 2 F E 1,...,E n x 1,...,x n E F, A 1,...,A n E x A E x 1,...,x n A 1,...,A n E S F A

36 S E x 1,...,x n E E f x 1,...,x n S S

37 T S P S M V M S 1 S n P V 1 V n S V T S M P V M

38 S 1, S 2 S 1 S 2 S 1 S 1 S 1,..., S n n S 1,..., S n

45 ( x i x x i ) z

46 x i x x i { x[t] i indices, t = argi indices z[i] = y[i] i / indices

47 Ax A x arg i indices y = Ax A x

48 A = y i i A ij x j j x j x

49 ˆ

55 Φ Φ Φ

58 n i i

59 m n [m, n], Unit

63 y = Ax x

66 f (x i, y i )= x i y i 2 X = {x 0, x 1,...,x n } Y = {y 0, y 1,...,y n }

68 x i y i x i y i m s ζ(s) P(k; s) = k s ζ(s), k > 0, s > 1 s s s m

69 s min max max min (ψ) ψ = W max W min W max W min ψ ψ = 1

71 m m ψm ψ ψ = 1

72 ψ = 1 ψ = 1 m > 50

73 ψ = 10 ψ = 100

74 ψ = 1000 ψ = 10000

75 ψ = m < 10 ψ = 1 ψ = m < 50 ψ = 100 ψ = 1000 ψ = ψ = ψ 1000

77 O P

78 P P { } { } G = N, E N n N n o, n h O P n o n n h n h E n i n j n i n j e E e b, e c, e d {False, True} C D e b e b e c n i e d n i C c C C = {None, Local, Global}

79 None Local local local local local Global global global D = {None, Backward, Forward} None None Backward n 1 0 Backward Forward 0 n 1 Forward Global None n n

80 (N, L,...,L) L (N,...,N) N (N, L, G) N (N, N, N) N (N, L) N (N, N, N) N (N, L) N (N, N, N) N (G, L) L (N, N) N (L, L, G) N (N, N, N) N (N, L,...,L) L (N,...,N) N (N, L, G) N (N, N, N) N (N, L) N (N, F, N) F (N, L) N (N, B, N) B (G, L) L (N, N) N (L, L, G) N (N, N, N) N

81 n n o n h n n e n n ec n ed n e c < n ec (n ed = None) (e d = n ed ) e b = True n n

82 k 2 k

94 !"#"$$$%&'(")&'*("$+&'**("$$*("$,&'("-&'***" "!./012.)"#"$'(")(",*" "!./012.%"#"$'(")("+("+("-*" "!./343"""#"$%&'(")&'("+&'(",&'("-&'*" [1, 2, 3] [1, 2, 3.0] [[1, 2], [3]] [[1, 2],3]

95 n m ψm (ψ + n 1)m ψmn ψ

96 3, 3, Unit [[1, 2, 3], [4, 5, 6], [7, 8, 9]] (3, 3) (3, 1) [1, 2, 3, 4, 5, 6, 7, 8, 9] (1, 3) [1, 4, 7, 2, 5, 8, 3, 6, 9] (1, 4) [1, 4, 7,, 2, 5, 8,, 3, 6, 9, ]

98 6.39s 0.52s 195µs 6.04s 0.49s 202µs 5.82s 0.49s 171µs 7.46s 0.63s 360µs

100

101 { } { }

102 A = M K K K M = 4 K = 3 K M

103

104 Ax = b A

105

106 A M 1 x 0 b n r 0 = b Ax 0 z 0 = M 1 r 0 p 0 = z 0 k = 0 rt k z k α k = pk T Ap k x k+1 = x k + α k p k r k+1 = r k α k Ap k z k+1 = M 1 r k+1 β k = zt k+1 r k+1 z T k r k p k+1 = z k+1 + β k p k k = k + 1 k > n M 1 a, b, c [ ] ai b m i = i [ ] 1 [ ] p i = m 1 ai b i = i 1 ci b = i b i c i a i c i b i b i b i a i b i c i

107 v i V v i V i v 1 n i V n i α i β i ff fi

108

109

110 A v 1 V i = 1 z i = Av i α i = z T i V i z i = z i VV T z i β i = z i V i+1 = z i /β i i = i + 1

111 T j j j α 1, α 2,...,α j β 1, β 2,...,β j 1 SΘS = T j

112 x R n y { 1, 1} x i, i {1,..., l} y i, i {1,..., l} max α F(α) = subject to l i=1 α i 1 2 αt Qα 0 α i C, i 1...l y T α = 0 x i R n i y i { 1, 1} x i α i C Q ij = y i y j Φ(x i, x j ) Φ(x i, x j ) Φ(x i, x j ; γ) =exp { γ x i x j 2} γ

113 α i x i y i i {1..l} τ α i f i = y i i {1..l} b high b low i high i low α ihigh α ilow f i i {1..l} b high i high b low i low α ihigh α ilow b low b high + 2τ b high = 1 i high = min{i : y i = 1} b low = 1 i low = min{i : y i = 1} i high i low α i low = α ilow + y ilow (b high b low )/η α i high = α ihigh + y ilow y ihigh (α ilow α i low ) η = Φ(x ihigh, x ihigh )+Φ(x ilow, x ilow ) 2Φ(x ihigh, x ilow ) α i low α i high 0 α i C f i = l j=1 α jy j Φ(x i, x j ) y i α f f i = f i +(α i high α ihigh )y ihigh Φ(x ihigh, x i ) +(α i low α ilow )y ilow Φ(x ilow, x i )

114 I high = {i :0< α i < C} {i : y i > 0, α i = 0} {i : y i < 0, α i = C} I low = {i :0< α i < C} {i : y i > 0, α i = C} {i : y i < 0, α i = 0} ɛ {i : ɛ < α i < (C ɛ)} b high = min{ f i : i I high } b low = max{ f i : i I low } b low b high + 2τ i high i low α i high = arg min{ f i : i I high } i low = arg max{ f i : i I low }

115

116

117

118

119

120

121 1 ψ ψ > % 100% 10% 33% 9

122 45 100%

123 {Distributed, Sequential}

124

125 45 100%

126

127

128

129

130

131

132

Fast Support Vector Machine Training and Classification on Graphics Processors

Fast Support Vector Machine Training and Classification on Graphics Processors Bryan Christopher Catanzaro Narayanan Sundaram Kurt Keutzer Electrical Engineering and Computer Sciences University of California