High Order Reverse Mode of AD Theory and Implementation
|
|
- Pierce Norris
- 7 years ago
- Views:
Transcription
1 Hgh Order Reverse Mode of AD Theory and Implementaton Mu Wang and Alex Pothen Department of Computer Scence Purdue Unversty September 30, 2016 Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
2 Research Overvew Second order reverse mode : More effcent n evaluatng Hessan n both complexty and memory usage n many applcatons. Proved to be equvalent to an varance of vertex elmnaton on the computatonal graph of the gradent Hgh order reverse mode : Hgh order reverse mode : evaluatng dervatve tensor d f up to any order n reverse mode Implementaton : ReverseAD Applcatons: Uncertanty quantfcaton Chemstry : exchange-correlaton (XC) energy functonal Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
3 Research Overvew Second order reverse mode : More effcent n evaluatng Hessan n both complexty and memory usage n many applcatons. Proved to be equvalent to an varance of vertex elmnaton on the computatonal graph of the gradent Hgh order reverse mode : Hgh order reverse mode : evaluatng dervatve tensor d f up to any order n reverse mode Implementaton : ReverseAD Applcatons: Uncertanty quantfcaton Chemstry : exchange-correlaton (XC) energy functonal Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
4 Research Overvew Second order reverse mode : More effcent n evaluatng Hessan n both complexty and memory usage n many applcatons. Proved to be equvalent to an varance of vertex elmnaton on the computatonal graph of the gradent Hgh order reverse mode : Hgh order reverse mode : evaluatng dervatve tensor d f up to any order n reverse mode Implementaton : ReverseAD Applcatons: Uncertanty quantfcaton Chemstry : exchange-correlaton (XC) energy functonal Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
5 Research Overvew Second order reverse mode : More effcent n evaluatng Hessan n both complexty and memory usage n many applcatons. Proved to be equvalent to an varance of vertex elmnaton on the computatonal graph of the gradent Hgh order reverse mode : Hgh order reverse mode : evaluatng dervatve tensor d f up to any order n reverse mode Implementaton : ReverseAD Applcatons: Uncertanty quantfcaton Chemstry : exchange-correlaton (XC) energy functonal Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
6 Research Overvew Second order reverse mode : More effcent n evaluatng Hessan n both complexty and memory usage n many applcatons. Proved to be equvalent to an varance of vertex elmnaton on the computatonal graph of the gradent Hgh order reverse mode : (ths talk) Hgh order reverse mode : evaluatng dervatve tensor d f up to any order n reverse mode Implementaton : ReverseAD Applcatons: Uncertanty quantfcaton Chemstry : exchange-correlaton (XC) energy functonal Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
7 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
8 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
9 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
10 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
11 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
12 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
13 Background For a scalar objectve functon f : R n R Frst order AD: Forward : [F1 f ](x, ẋ) = f x ẋ = f T ẋ Reverse : [R1 f ](x) = ( f x 1,, f x 1 ) = f Second order AD: (Pure) Forward : [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ Mxed : [R1 F 1 f ](x, ẋ) = 2 f ẋ (Pure) Reverse : [R2 f ](x) = 2 f Hgh order AD: (Pure) Forward : Hgh order taylor coeffcents (Pure) Reverse : Hgh order reverse mode Mxed modes then can be generated Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
14 Hgh Order Forward Mode Accumulate hgh order taylor coeffcents 1 : d f : d-th order dervatve tensor (symmetrc). d f ẋ : A tensor-vector product, (d 1)-th order symmetrc tensor [ [[. d f ẋ] ẋ ] ] ẋ : A scalar, the d-th order taylor coeffcents. 1 Grewank, Andreas, Jean Utke, and Andrea Walther. Evaluatng hgher dervatve tensors by forward propagaton of unvarate Taylor seres. Mathematcs of Computaton of the Amercan Mathematcal Socety (2000): Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
15 Hgh Order Forward Mode Accumulate hgh order taylor coeffcents 1 : d f : d-th order dervatve tensor (symmetrc). d f ẋ : A tensor-vector product, (d 1)-th order symmetrc tensor [ [[. d f ẋ] ẋ ] ] ẋ : A scalar, the d-th order taylor coeffcents. d = 2: [F 2 f ](x, ẋ) = 1 2ẋT 2 f ẋ [ 2 f ] j = [F 2 f ](x, e + e j ) [F 2 f ](x, e ) [F 2 f ](x, e j ) 1 Grewank, Andreas, Jean Utke, and Andrea Walther. Evaluatng hgher dervatve tensors by forward propagaton of unvarate Taylor seres. Mathematcs of Computaton of the Amercan Mathematcal Socety (2000): Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
16 Hgh Order Forward Mode Accumulate hgh order taylor coeffcents 1 : d f : d-th order dervatve tensor (symmetrc). d f ẋ : A tensor-vector product, (d 1)-th order symmetrc tensor [ [[. d f ẋ] ẋ ] ] ẋ : A scalar, the d-th order taylor coeffcents. General case: [F d f ](x, ẋ) = 1 d![ [[ d f ẋ] ẋ ] [ d f ] 1 d : a lnear combnaton of ] ẋ {[F d f ](x, ė) : ė Span{e 1,, e d }}} 1 Grewank, Andreas, Jean Utke, and Andrea Walther. Evaluatng hgher dervatve tensors by forward propagaton of unvarate Taylor seres. Mathematcs of Computaton of the Amercan Mathematcal Socety (2000): Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
17 Hgh Order Forward Mode Accumulate hgh order taylor coeffcents 1 : d f : d-th order dervatve tensor (symmetrc). d f ẋ : A tensor-vector product, (d 1)-th order symmetrc tensor [ [[. d f ẋ] ẋ ] ] ẋ : A scalar, the d-th order taylor coeffcents. General case: [F d f ](x, ẋ) = 1 d![ [[ d f ẋ] ẋ ] [ d f ] 1 d : a lnear combnaton of ] ẋ {[F d f ](x, ė) : ė Span{e 1,, e d }}} Complexty : O( ( (n+d 1)) d l) 1 Grewank, Andreas, Jean Utke, and Andrea Walther. Evaluatng hgher dervatve tensors by forward propagaton of unvarate Taylor seres. Mathematcs of Computaton of the Amercan Mathematcal Socety (2000): Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
18 Reverse Mode Revst Defnton After process the SAC v = ϕ (v j ) {vj :v j v } n reverse mode, the process SACs defne an equvalent functon f (S ). The objectve functon s the composton of f and the remanng SACs and S s the current lve varable set. Observaton reverse mode computes the dervatves of f (S ) n each step by followng the order chan rule. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
19 Reverse Mode Revst Defnton After process the SAC v = ϕ (v j ) {vj :v j v } n reverse mode, the process SACs defne an equvalent functon f (S ). The objectve functon s the composton of f and the remanng SACs and S s the current lve varable set. Observaton Second order reverse mode computes the frst and the second order dervatves of f (S ) n each step by followng the frst and second order chan rule. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
20 Reverse Mode Revst Defnton After process the SAC v = ϕ (v j ) {vj :v j v } n reverse mode, the process SACs defne an equvalent functon f (S ). The objectve functon s the composton of f and the remanng SACs and S s the current lve varable set. Observaton Second order Hgh order reverse mode computes the frst and the second order dervatves up to order d of f (S ) n each step by followng the frst and second hgh order chan rule. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
21 Hgh Order Chan Rule Observaton Hgh order reverse mode computes the dervatves up to order d of f n each step by followng the hgh order chan rule. When process v = ϕ (v j ) {vj :v j v }: S = S +1 \ {v } {v j : v j v } f (S ) = f +1 (S +1 \ {v }, v = ϕ (v j ) {vj :v j v }) Hgh order chan rule: dervatves of f +1 (S +1 ) dervatves of f (S ) General case of Faà d Bruno equaton Specal case of the equaton n Ma, Ma, Tsoy-Wo. Hgher chan formula proved by combnatorcs. the electronc journal of combnatorcs 16.1 (2009): N21. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
22 Hgh Order Chan Rule Observaton Hgh order reverse mode computes the dervatves up to order d of f n each step by followng the hgh order chan rule. When process v = ϕ (v j ) {vj :v j v }: S = S +1 \ {v } {v j : v j v } f (S ) = f +1 (S +1 \ {v }, v = ϕ (v j ) {vj :v j v }) Hgh order chan rule: dervatves of f +1 (S +1 ) dervatves of f (S ) General case of Faà d Bruno equaton Specal case of the equaton n Ma, Ma, Tsoy-Wo. Hgher chan formula proved by combnatorcs. the electronc journal of combnatorcs 16.1 (2009): N21. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
23 Hgh Order Chan Rule Multset A multset D s a generalzaton of the noton of a set n whch members are allowed to appear more than once. We use D S to represent the famly of all multsets over S. That s: D S = {D : D = {e 1, e 2,, e d }, e S, 1 d} Dervatve Mappng For a functon f (S), ts order d dervatve tensor can be represented as a mappng from D D S, D = d to R as: T f (D) = D f D = D f v 1 v 2 v D Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
24 Hgh Order Chan Rule Multset A multset D s a generalzaton of the noton of a set n whch members are allowed to appear more than once. We use D S to represent the famly of all multsets over S. That s: D S = {D : D = {e 1, e 2,, e d }, e S, 1 d} Dervatve Mappng For a functon f (S), ts order d dervatve tensor can be represented as a mappng from D D S, D = d to R as: T f (D) = D f D = D f v 1 v 2 v D Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
25 Hgh Order Chan Rule Multset A multset D s a generalzaton of the noton of a set n whch members are allowed to appear more than once. We use D S to represent the famly of all multsets over S. That s: D S = {D : D = {e 1, e 2,, e d }, e S, 1 d} Dervatve Mappng For a functon f (S), ts order d dervatve tensor can be represented as a mappng from D D S, D = d to R as: T f (D) = D f D = D f v 1 v 2 v D Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
26 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Frst order : D = {v} a (v) = a +1 (v) + v a +1(v ) v a +1(v ) : D L =, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
27 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Frst order : D = {v} a (v) = a +1 (v) + v a +1(v ) v a +1(v ) : D L =, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
28 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Frst order : D = {v} a (v) = a +1 (v) + v a +1(v ) v a +1(v ) : D L =, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
29 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Frst order : D = {v} a (v) = a +1 (v) + v a +1(v ) v a +1(v ) : D L =, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
30 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Frst order : D = {v} a (v) = a +1 (v) + v a +1(v ) v a +1(v ) : D L =, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
31 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
32 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
33 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) v h +1(v, u) : D L = {u}, D 1 = {v} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
34 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) u h +1(v, v ) : D L = {v}, D 1 = {u} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
35 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) v u h +1(v, v ) : D L =, D 1 = {v}, D 2 = {u} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
36 Hgh Order Chan Rule T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D. Second order : D = {v, u} h (v, u) = h +1 (v, u) + v h +1(v, u) + u h +1(v, v ) + v u h +1(v, v ) + 2 ϕ v u a +1(v ) 2 ϕ v u a +1(v ) : D L =, D 1 = {v, u} Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
37 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
38 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
39 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
40 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
41 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
42 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
43 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
44 Hgh Order Reverse Mode : Complexty T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L D L, D 1,, D r s a partton of D. B d+1 summatons : (d + 1) th Bell number. O(B d+1 s d 1 ) updates for each SAC. Overall complexty : O(B d+1 s d 1 l), s = max{s } ϕ ] T f+1 (D L {v r }) D 1 D r When d = 1 : O(l) Baur-Strassen theorem. When d = 2 : O(l s) second order reverse mode When d = 3 : O(l s 2 ) thrd order reverse mode. Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
45 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (D L, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
46 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (D L, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
47 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (DL, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
48 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (DL, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
49 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (DL, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
50 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (DL, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
51 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r D L, D 1,, D r s a partton of D Generate all D L, s.t, T f+1 (D L {v r }) 0 and D 1,, D r, s.t, 0, 1 r D Then perform ncremental updates on D = D L D 1 D r More than one way to partton D nto D L, D 1,, D r. SymCoeff (DL, D 1,, D r ) : Multplcty that partton D nto D L, D 1,, D r. Flat code for pre-computed symmetrc coeffcents 5k lnes of generated code for up to sxth order Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
52 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r ReverseAD : an operator overloadng mplementaton of the hgh order reverse mode n C++11. Avalable at Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
53 Hgh Order Reverse Mode : Implementaton T f (D) = T f+1 (D) + D L D [ D D j = D 1 D r =D\D L ϕ ] T f+1 (D L {v r }) D 1 D r ReverseAD : an operator overloadng mplementaton of the hgh order reverse mode n C++11. Avalable at Monotonc ndexng for varables on the trace v j v = ndex(v j ) < ndex(v ) Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
54 Performance : Synthetc Functon A synthetc functon desgned wth parameters: n : number of ndependent varables s : sze of lve varables durng the functon evaluaton l : the complexty of the functon Dense dervatves z z, s z 2.0, y = t, =1 z , ID(z) = log(exp(z)), t = ID k ID 1 (z ), z = t, t = n x. =1 1.0/(1.0/z), sn(asn(z)). Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
55 Performance : Synthetc Functon A synthetc functon desgned wth parameters: n : number of ndependent varables s : sze of lve varables durng the functon evaluaton l : the complexty of the functon Dense dervatves z z, s z 2.0, y = t, =1 z , ID(z) = log(exp(z)), t = ID k ID 1 (z ), z = t, t = n x. =1 1.0/(1.0/z), sn(asn(z)). Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
56 Performance : Synthetc Functon Fxed l, let n and s change smultaneously Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
57 Performance : Synthetc Functon Fxed l, let n and s change smultaneously Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
58 Performance : Synthetc Functon Fxed l and n, changed s Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
59 Performance : Synthetc Functon Fxed l and n, changed s Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
60 Performance : Synthetc Functon Fxed l and s, changed n Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
61 Performance : Synthetc Functon Fxed l and s, changed n Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
62 Applcaton : XCFUN (on gong) Arbtrary order Exchange-Correlaton functonal lbrary Usng lbtaylor to evaluate dervatves of functonals Up to thrd order n current mplementaton Small number of ndependents : 20 at most Not so-complex functonals On a collecton of functonals: Thrd order Lbtaylor : 81ms Thrd order ReverseAD : 20ms Fourth order ReverseAD : 83ms Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
63 Applcaton : XCFUN (on gong) Arbtrary order Exchange-Correlaton functonal lbrary Usng lbtaylor to evaluate dervatves of functonals Up to thrd order n current mplementaton Small number of ndependents : 20 at most Not so-complex functonals On a collecton of functonals: Thrd order Lbtaylor : 81ms Thrd order ReverseAD : 20ms Fourth order ReverseAD : 83ms Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
64 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ d f ẋ] ẋ ] ] ẋ Fd : The structural (and sparsty) propertes of Tf determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
65 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ d f ẋ] ẋ ] ] ẋ Fd : The structural (and sparsty) propertes of Tf determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
66 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ F d : d f ẋ] ẋ ] ] ẋ The structural (and sparsty) propertes of T f determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
67 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ F d : d f ẋ] ẋ ] ] ẋ The structural (and sparsty) propertes of T f determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
68 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ F d : d f ẋ] ẋ ] ] ẋ The structural (and sparsty) propertes of T f determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
69 Concluson and Future Work Hgh order dervatve tensors could (and probably should) be drectly evaluated va reverse mode. A seres of algorthms to evaluate dervatves T f up to order d : F d F 1 R d 1 R d R d : symmetrc dervatve tensor d f F 1 R[ d 1 : tensor-vector d f ẋ [[ F d : d f ẋ] ẋ ] ] ẋ The structural (and sparsty) propertes of T f determnes the optmal method. General compresson and recovery usng F1 R d 1. perfectly parallelzable Mu Wang and Alex Pothen Hgh Order Reverse AD September 30, / 1
BERNSTEIN POLYNOMIALS
On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More informationCausal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting
Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of
More informationProject Networks With Mixed-Time Constraints
Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
More informationLoop Parallelization
- - Loop Parallelzaton C-52 Complaton steps: nested loops operatng on arrays, sequentell executon of teraton space DECLARE B[..,..+] FOR I :=.. FOR J :=.. I B[I,J] := B[I-,J]+B[I-,J-] ED FOR ED FOR analyze
More informationLuby s Alg. for Maximal Independent Sets using Pairwise Independence
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationwhere the coordinates are related to those in the old frame as follows.
Chapter 2 - Cartesan Vectors and Tensors: Ther Algebra Defnton of a vector Examples of vectors Scalar multplcaton Addton of vectors coplanar vectors Unt vectors A bass of non-coplanar vectors Scalar product
More informationConversion between the vector and raster data structures using Fuzzy Geographical Entities
Converson between the vector and raster data structures usng Fuzzy Geographcal Enttes Cdála Fonte Department of Mathematcs Faculty of Scences and Technology Unversty of Combra, Apartado 38, 3 454 Combra,
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationRecurrence. 1 Definitions and main statements
Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.
More informationThe Development of Web Log Mining Based on Improve-K-Means Clustering Analysis
The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationLeast Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2016. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng
More informationGeneralizing the degree sequence problem
Mddlebury College March 2009 Arzona State Unversty Dscrete Mathematcs Semnar The degree sequence problem Problem: Gven an nteger sequence d = (d 1,...,d n ) determne f there exsts a graph G wth d as ts
More informationApplication of Quasi Monte Carlo methods and Global Sensitivity Analysis in finance
Applcaton of Quas Monte Carlo methods and Global Senstvty Analyss n fnance Serge Kucherenko, Nlay Shah Imperal College London, UK skucherenko@mperalacuk Daro Czraky Barclays Captal DaroCzraky@barclayscaptalcom
More informationTesting and Debugging Resource Allocation for Fault Detection and Removal Process
Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): 93-00 The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: 0-9085) Testng and Debuggng Resource Allocaton
More informationRing structure of splines on triangulations
www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationA fast method for binary programming using first-order derivatives, with application to topology optimization with buckling constraints
INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING Int. J. Numer. Meth. Engng (2012) Publshed onlne n Wley Onlne Lbrary (wleyonlnelbrary.com)..4367 A fast method for bnary programmng usng frst-order
More informationThe Greedy Method. Introduction. 0/1 Knapsack Problem
The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton
More informationThe Mathematical Derivation of Least Squares
Pscholog 885 Prof. Federco The Mathematcal Dervaton of Least Squares Back when the powers that e forced ou to learn matr algera and calculus, I et ou all asked ourself the age-old queston: When the hell
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems
More informationSPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:
SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and
More informationLinear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits
Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.
More informationPOLYSA: A Polynomial Algorithm for Non-binary Constraint Satisfaction Problems with and
POLYSA: A Polynomal Algorthm for Non-bnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n
More informationAdaptive Fractal Image Coding in the Frequency Domain
PROCEEDINGS OF INTERNATIONAL WORKSHOP ON IMAGE PROCESSING: THEORY, METHODOLOGY, SYSTEMS AND APPLICATIONS 2-22 JUNE,1994 BUDAPEST,HUNGARY Adaptve Fractal Image Codng n the Frequency Doman K AI UWE BARTHEL
More informationCalculation of Sampling Weights
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample
More informationLogistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationEnabling P2P One-view Multi-party Video Conferencing
Enablng P2P One-vew Mult-party Vdeo Conferencng Yongxang Zhao, Yong Lu, Changja Chen, and JanYn Zhang Abstract Mult-Party Vdeo Conferencng (MPVC) facltates realtme group nteracton between users. Whle P2P
More information+ + + - - This circuit than can be reduced to a planar circuit
MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationSIMPLE LINEAR CORRELATION
SIMPLE LINEAR CORRELATION Smple lnear correlaton s a measure of the degree to whch two varables vary together, or a measure of the ntensty of the assocaton between two varables. Correlaton often s abused.
More informationLogical Development Of Vogel s Approximation Method (LD-VAM): An Approach To Find Basic Feasible Solution Of Transportation Problem
INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME, ISSUE, FEBRUARY ISSN 77-866 Logcal Development Of Vogel s Approxmaton Method (LD- An Approach To Fnd Basc Feasble Soluton Of Transportaton
More informationSUMMARY. Topology optimization, buckling, eigenvalue, derivative, structural optimization 1. INTRODUCTION
INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING Int. J. Numer. Meth. Engng 0000; 00:1 20 Publshed onlne n Wley InterScence (www.nterscence.wley.com). A fast method for solvng bnary programmng
More information1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)
6.3 / -- Communcaton Networks II (Görg) SS20 -- www.comnets.un-bremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes
More informationA Prefix Code Matching Parallel Load-Balancing Method for Solution-Adaptive Unstructured Finite Element Graphs on Distributed Memory Multicomputers
Ž. The Journal of Supercomputng, 15, 25 49 2000 2000 Kluwer Academc Publshers. Manufactured n The Netherlands. A Prefx Code Matchng Parallel Load-Balancng Method for Soluton-Adaptve Unstructured Fnte Element
More informationHow To Understand The Results Of The German Meris Cloud And Water Vapour Product
Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller
More informationAn artificial Neural Network approach to monitor and diagnose multi-attribute quality control processes. S. T. A. Niaki*
Journal of Industral Engneerng Internatonal July 008, Vol. 4, No. 7, 04 Islamc Azad Unversty, South Tehran Branch An artfcal Neural Network approach to montor and dagnose multattrbute qualty control processes
More informationA machine vision approach for detecting and inspecting circular parts
A machne vson approach for detectng and nspectng crcular parts Du-Mng Tsa Machne Vson Lab. Department of Industral Engneerng and Management Yuan-Ze Unversty, Chung-L, Tawan, R.O.C. E-mal: edmtsa@saturn.yzu.edu.tw
More informationRate Monotonic (RM) Disadvantages of cyclic. TDDB47 Real Time Systems. Lecture 2: RM & EDF. Priority-based scheduling. States of a process
Dsadvantages of cyclc TDDB47 Real Tme Systems Manual scheduler constructon Cannot deal wth any runtme changes What happens f we add a task to the set? Real-Tme Systems Laboratory Department of Computer
More informationRisk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008
Rsk-based Fatgue Estmate of Deep Water Rsers -- Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More informationImplementation of Deutsch's Algorithm Using Mathcad
Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationSolution: Let i = 10% and d = 5%. By definition, the respective forces of interest on funds A and B are. i 1 + it. S A (t) = d (1 dt) 2 1. = d 1 dt.
Chapter 9 Revew problems 9.1 Interest rate measurement Example 9.1. Fund A accumulates at a smple nterest rate of 10%. Fund B accumulates at a smple dscount rate of 5%. Fnd the pont n tme at whch the forces
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationAPPLICATION OF PROBE DATA COLLECTED VIA INFRARED BEACONS TO TRAFFIC MANEGEMENT
APPLICATION OF PROBE DATA COLLECTED VIA INFRARED BEACONS TO TRAFFIC MANEGEMENT Toshhko Oda (1), Kochro Iwaoka (2) (1), (2) Infrastructure Systems Busness Unt, Panasonc System Networks Co., Ltd. Saedo-cho
More informationData Broadcast on a Multi-System Heterogeneous Overlayed Wireless Network *
JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, 819-840 (2008) Data Broadcast on a Mult-System Heterogeneous Overlayed Wreless Network * Department of Computer Scence Natonal Chao Tung Unversty Hsnchu,
More informationFast degree elevation and knot insertion for B-spline curves
Computer Aded Geometrc Desgn 22 (2005) 183 197 www.elsever.com/locate/cagd Fast degree elevaton and knot nserton for B-splne curves Q-Xng Huang a,sh-mnhu a,, Ralph R. Martn b a Department of Computer Scence
More informationAn Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems
STAN-CS-73-355 I SU-SE-73-013 An Analyss of Central Processor Schedulng n Multprogrammed Computer Systems (Dgest Edton) by Thomas G. Prce October 1972 Techncal Report No. 57 Reproducton n whole or n part
More informationFisher Markets and Convex Programs
Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and
More informationCompiling for Parallelism & Locality. Dependence Testing in General. Algorithms for Solving the Dependence Problem. Dependence Testing
Complng for Parallelsm & Localty Dependence Testng n General Assgnments Deadlne for proect 4 extended to Dec 1 Last tme Data dependences and loops Today Fnsh data dependence analyss for loops General code
More informationTHE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES
The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered
More informationProduction. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.
Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set
More informationVariance estimation for the instrumental variables approach to measurement error in generalized linear models
he Stata Journal (2003) 3, Number 4, pp. 342 350 Varance estmaton for the nstrumental varables approach to measurement error n generalzed lnear models James W. Hardn Arnold School of Publc Health Unversty
More informationOverview of monitoring and evaluation
540 Toolkt to Combat Traffckng n Persons Tool 10.1 Overvew of montorng and evaluaton Overvew Ths tool brefly descrbes both montorng and evaluaton, and the dstncton between the two. What s montorng? Montorng
More informationDescriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications
CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary
More informationA hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm
Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel
More informationRisk Model of Long-Term Production Scheduling in Open Pit Gold Mining
Rsk Model of Long-Term Producton Schedulng n Open Pt Gold Mnng R Halatchev 1 and P Lever 2 ABSTRACT Open pt gold mnng s an mportant sector of the Australan mnng ndustry. It uses large amounts of nvestments,
More information8 Algorithm for Binary Searching in Trees
8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the
More informationImplementation of Boolean Functions through Multiplexers with the Help of Shannon Expansion Theorem
Internatonal Journal o Computer pplcatons (975 8887) Volume 62 No.6, January 23 Implementaton o Boolean Functons through Multplexers wth the Help o Shannon Expanson Theorem Saurabh Rawat Graphc Era Unversty.
More informationAdaptive Clinical Trials Incorporating Treatment Selection and Evaluation: Methodology and Applications in Multiple Sclerosis
Adaptve Clncal Trals Incorporatng Treatment electon and Evaluaton: Methodology and Applcatons n Multple cleross usan Todd, Tm Frede, Ngel tallard, Ncholas Parsons, Elsa Valdés-Márquez, Jeremy Chataway
More informationAn interactive system for structure-based ASCII art creation
An nteractve system for structure-based ASCII art creaton Katsunor Myake Henry Johan Tomoyuk Nshta The Unversty of Tokyo Nanyang Technologcal Unversty Abstract Non-Photorealstc Renderng (NPR), whose am
More informationExtending Probabilistic Dynamic Epistemic Logic
Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set
More informationProbability and Optimization Models for Racing
1 Probablty and Optmzaton Models for Racng Vctor S. Y. Lo Unversty of Brtsh Columba Fdelty Investments Dsclamer: Ths presentaton does not reflect the opnons of Fdelty Investments. The work here was completed
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationAn Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services
An Evaluaton of the Extended Logstc, Smple Logstc, and Gompertz Models for Forecastng Short Lfecycle Products and Servces Charles V. Trappey a,1, Hsn-yng Wu b a Professor (Management Scence), Natonal Chao
More informationOnline Inference of Topics with Latent Dirichlet Allocation
Onlne Inference of Topcs wth Latent Drchlet Allocaton Kevn R. Cann Computer Scence Dvson Unversty of Calforna Berkeley, CA 94720 kevn@cs.berkeley.edu Le Sh Helen Wlls Neuroscence Insttute Unversty of Calforna
More informationAn MILP model for planning of batch plants operating in a campaign-mode
An MILP model for plannng of batch plants operatng n a campagn-mode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafe-concet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño
More informationBinomial Link Functions. Lori Murray, Phil Munz
Bnomal Lnk Functons Lor Murray, Phl Munz Bnomal Lnk Functons Logt Lnk functon: ( p) p ln 1 p Probt Lnk functon: ( p) 1 ( p) Complentary Log Log functon: ( p) ln( ln(1 p)) Motvatng Example A researcher
More informationUsing Series to Analyze Financial Situations: Present Value
2.8 Usng Seres to Analyze Fnancal Stuatons: Present Value In the prevous secton, you learned how to calculate the amount, or future value, of an ordnary smple annuty. The amount s the sum of the accumulated
More information"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *
Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC
More informationO(n) mass matrix inversion for serial manipulators and polypeptide chains using Lie derivatives Kiju Lee, Yunfeng Wang and Gregory S.
Robotca 7) volume 5, pp 739 75 7 Cambrdge Unversty Press do:7/s6357477385 Prnted n the Unted Kngdom On) mass matrx nverson for seral manpulators and polypeptde chans usng Le dervatves Ku Lee, Yunfeng Wang
More informationResearch Article Enhanced Two-Step Method via Relaxed Order of α-satisfactory Degrees for Fuzzy Multiobjective Optimization
Hndaw Publshng Corporaton Mathematcal Problems n Engneerng Artcle ID 867836 pages http://dxdoorg/055/204/867836 Research Artcle Enhanced Two-Step Method va Relaxed Order of α-satsfactory Degrees for Fuzzy
More informationDynamic Pricing for Smart Grid with Reinforcement Learning
Dynamc Prcng for Smart Grd wth Renforcement Learnng Byung-Gook Km, Yu Zhang, Mhaela van der Schaar, and Jang-Won Lee Samsung Electroncs, Suwon, Korea Department of Electrcal Engneerng, UCLA, Los Angeles,
More informationPSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12
14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationLearning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing
Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part
More informationECONOMICS OF PLANT ENERGY SAVINGS PROJECTS IN A CHANGING MARKET Douglas C White Emerson Process Management
ECONOMICS OF PLANT ENERGY SAVINGS PROJECTS IN A CHANGING MARKET Douglas C Whte Emerson Process Management Abstract Energy prces have exhbted sgnfcant volatlty n recent years. For example, natural gas prces
More informationCopulas. Modeling dependencies in Financial Risk Management. BMI Master Thesis
Copulas Modelng dependences n Fnancal Rsk Management BMI Master Thess Modelng dependences n fnancal rsk management Modelng dependences n fnancal rsk management 3 Preface Ths paper has been wrtten as part
More informationAn Integrated Semantically Correct 2.5D Object Oriented TIN. Andreas Koch
An Integrated Semantcally Correct 2.5D Object Orented TIN Andreas Koch Unverstät Hannover Insttut für Photogrammetre und GeoInformaton Contents Introducton Integraton of a DTM and 2D GIS data Semantcs
More information1. Math 210 Finite Mathematics
1. ath 210 Fnte athematcs Chapter 5.2 and 5.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210
More informationEconomic Interpretation of Regression. Theory and Applications
Economc Interpretaton of Regresson Theor and Applcatons Classcal and Baesan Econometrc Methods Applcaton of mathematcal statstcs to economc data for emprcal support Economc theor postulates a qualtatve
More informationControl Charts with Supplementary Runs Rules for Monitoring Bivariate Processes
Control Charts wth Supplementary Runs Rules for Montorng varate Processes Marcela. G. Machado *, ntono F.. Costa * * Producton Department, Sao Paulo State Unversty, Campus of Guaratnguetá, 56-4 Guaratnguetá,
More informationMultiple stage amplifiers
Multple stage amplfers Ams: Examne a few common 2-transstor amplfers: -- Dfferental amplfers -- Cascode amplfers -- Darlngton pars -- current mrrors Introduce formal methods for exactly analysng multple
More informationOn fourth order simultaneously zero-finding method for multiple roots of complex polynomial equations 1
General Mathematcs Vol. 6, No. 3 (2008), 9 3 On fourth order smultaneously zero-fndng method for multple roots of complex polynomal euatons Nazr Ahmad Mr and Khald Ayub Abstract In ths paper, we present
More informationAnswer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy
4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.
More informationA Crossplatform ECG Compression Library for Mobile HealthCare Services
A Crossplatform ECG Compresson Lbrary for Moble HealthCare Servces Alexander Borodn, Yulya Zavyalova Department of Computer Scence Petrozavodsk State Unversty Petrozavodsk, Russa {aborod, yzavyalo}@cs.petrsu.ru
More informationFinite difference method
grd ponts x = mesh sze = X NÜÆ Fnte dfference method Prncple: dervatves n the partal dfferental eqaton are approxmated by lnear combnatons of fncton vales at the grd ponts 1D: Ω = (0, X), (x ), = 0,1,...,
More informationAn Efficient and Simplified Model for Forecasting using SRM
HAFIZ MUHAMMAD SHAHZAD ASIF*, MUHAMMAD FAISAL HAYAT*, AND TAUQIR AHMAD* RECEIVED ON 15.04.013 ACCEPTED ON 09.01.014 ABSTRACT Learnng form contnuous fnancal systems play a vtal role n enterprse operatons.
More informationCourse outline. Financial Time Series Analysis. Overview. Data analysis. Predictive signal. Trading strategy
Fnancal Tme Seres Analyss Patrck McSharry patrck@mcsharry.net www.mcsharry.net Trnty Term 2014 Mathematcal Insttute Unversty of Oxford Course outlne 1. Data analyss, probablty, correlatons, vsualsaton
More informationAlternate Approximation of Concave Cost Functions for
Alternate Approxmaton of Concave Cost Functons for Process Desgn and Supply Chan Optmzaton Problems Dego C. Cafaro * and Ignaco E. Grossmann INTEC (UNL CONICET), Güemes 3450, 3000 Santa Fe, ARGENTINA Department
More informationA Study on Secure Data Storage Strategy in Cloud Computing
Journal of Convergence Informaton Technology Volume 5, Number 7, Setember 00 A Study on Secure Data Storage Strategy n Cloud Comutng Danwe Chen, Yanjun He, Frst Author College of Comuter Technology, Nanjng
More informationA generalized hierarchical fair service curve algorithm for high network utilization and link-sharing
Computer Networks 43 (2003) 669 694 www.elsever.com/locate/comnet A generalzed herarchcal far servce curve algorthm for hgh network utlzaton and lnk-sharng Khyun Pyun *, Junehwa Song, Heung-Kyu Lee Department
More informationPrediction of Disability Frequencies in Life Insurance
Predcton of Dsablty Frequences n Lfe Insurance Bernhard Köng Fran Weber Maro V. Wüthrch October 28, 2011 Abstract For the predcton of dsablty frequences, not only the observed, but also the ncurred but
More informationPRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB.
PRACTICE 1: MUTUAL FUNDS EVALUATION USING MATLAB. INDEX 1. Load data usng the Edtor wndow and m-fle 2. Learnng to save results from the Edtor wndow. 3. Computng the Sharpe Rato 4. Obtanng the Treynor Rato
More informationSensor placement for leak detection and location in water distribution networks
Sensor placement for leak detecton and locaton n water dstrbuton networks ABSTRACT R. Sarrate*, J. Blesa, F. Near, J. Quevedo Automatc Control Department, Unverstat Poltècnca de Catalunya, Rambla de Sant
More informationA Fast Incremental Spectral Clustering for Large Data Sets
2011 12th Internatonal Conference on Parallel and Dstrbuted Computng, Applcatons and Technologes A Fast Incremental Spectral Clusterng for Large Data Sets Tengteng Kong 1,YeTan 1, Hong Shen 1,2 1 School
More informationComparison of Control Strategies for Shunt Active Power Filter under Different Load Conditions
Comparson of Control Strateges for Shunt Actve Power Flter under Dfferent Load Condtons Sanjay C. Patel 1, Tushar A. Patel 2 Lecturer, Electrcal Department, Government Polytechnc, alsad, Gujarat, Inda
More informationQuantization Effects in Digital Filters
Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value
More informationEfficient Project Portfolio as a tool for Enterprise Risk Management
Effcent Proect Portfolo as a tool for Enterprse Rsk Management Valentn O. Nkonov Ural State Techncal Unversty Growth Traectory Consultng Company January 5, 27 Effcent Proect Portfolo as a tool for Enterprse
More information