WITH the increase in scale and complexity of process

Size: px
Start display at page:

Download "WITH the increase in scale and complexity of process"

Transcription

1 2052 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 Drect Causalty Detecton va the Transfer Entropy Approach Png Duan, Fan Yang, Member, IEEE, Tongwen Chen, Fellow, IEEE, and Srsh L. Shah, Member, IEEE Abstract The detecton of drect causalty, as opposed to ndrect causalty, s an mportant and challengng problem n root cause and hazard propagaton analyss. Several methods provde effectve solutons to ths problem when lnear relatonshps between varables are nvolved. For nonlnear relatonshps, currently only overall causalty analyss can be conducted, but drect causalty cannot be dentfed for such processes. In ths paper, we descrbe a drect causalty detecton approach sutable for both lnear and nonlnear connectons. Based on an extenson of the transfer entropy approach, a drect transfer entropy (DTE concept s proposed to detect whether there s a drect nformaton flow pathway from one varable to another. Especally, a dfferental drect transfer entropy concept s defned for contnuous random varables, and a normalzaton method for the dfferental drect transfer entropy s presented to determne the connectvty strength of drect causalty. The effectveness of the proposed method s llustrated by several examples, ncludng one expermental case study and one ndustral case study. Index Terms Dfferental transfer entropy, drect causalty, drect transfer entropy (DTE, nformaton flow pathway, kernel estmaton, normalzaton. NTE x ỹ Normalzed dscrete transfer entropy (NTE dsc from x to ỹ. NTE c x y Normalzed dfferental transfer entropy (NTE dff from x to y. NDTE c x y Normalzed dfferental drect transfer Entropy (NDTE dff from x to y. Eq. (1: Defnton of the TE dff from x to y. Eq. (2: Defnton of the TE dff from x to z. Eq. (3: Defnton of the TE dff from z to y. Eq. (4: Defnton of the DTE dff from x to y. Eq. (5: Defnton of the DTE dff from z to y. Eq. (6: Defnton of the TE dsc from x to ỹ. Eq. (10: Relatonshp between the TE dff from x to y and the TE dsc from x to ỹ. Eq. (11: Defnton of the DTE dsc from x to ỹ. Eq. (18: Lnear normalzaton of the TE dff. Eq. (19: Nonlnear normalzaton of the TE dff. Eq. (20: Normalzaton of the DTE dff. Eq. (21: Defnton of the DTE dff from x to y wth multple ntermedate varables. NOMENCLATURE x, y, z Contnuous random varables. x, ỹ, z Quantzed x, quantzed y, and quantzed z wth quantzaton bn szes x, y, and z, respectvely. T x y Dfferental transfer entropy(te dff from x to y. D x y Dfferental drect transfer entropy (DTE dff from x to y. t x ỹ Dscrete transfer entropy (TE dsc from x to ỹ. d x ỹ Dscrete drect transfer entropy (DTE dsc from x to ỹ. Manuscrpt receved May 2, 2012; revsed October 11, 2012; accepted December 2, Manuscrpt receved n fnal form December 7, Date of publcaton January 9, 2013; date of current verson October 15, Ths work was supported by an NSERC Strategc Project, an NSFC Project under Grant and the Tsnghua Natonal Laboratory for Informaton Scence and Technology Cross-Dscplne Foundaton. Recommended by Assocate Edtor J. Yu. P. Duan and T. Chen are wth the Department of Electrcal and Computer Engneerng, Unversty of Alberta, Edmonton, AB T6G 2V4, Canada (e-mal: F. Yang s wth the Department of Automaton, Tsnghua Natonal Laboratory for Informaton Scence and Technology, Tsnghua Unversty, Bejng , Chna (e-mal: S. L. Shah s wth the Department of Chemcal and Materals Engneerng, Unversty of Alberta, Edmonton, AB T6G 2G6, Canada (e-mal: Dgtal Object Identfer /TCST IEEE I. INTRODUCTION WITH the ncrease n scale and complexty of process operatons, the detecton and dagnoss of plantwde abnormaltes and dsturbances are major problems n the process ndustry. Compared wth the tradtonal fault detecton, fault detecton and dagnoss n a large-scale complex system are partcularly challengng because of the hgh degree of nterconnectons among dfferent parts n the system. A smple fault may easly propagate along nformaton and materal flow pathways and affect other parts of the system. To determne the root cause(s of certan abnormalty, t s mportant to capture the process connectvty and fnd the connectng pathways. A qualtatve process model n the form of a dgraph has been wdely used n root cause and hazard propagaton analyss [1]. Dgraph-based models usually express the causal relatonshps between faults and symptoms and defne the fault propagaton pathways by ncorporatng expert knowledge of the process [2]. A drawback s that extractng expert knowledge s very tme consumng and that knowledge s not always easly avalable [3]. The modelng of dgraphs can also be based on mathematcal equatons [4], [5], yet for largescale complex processes t s dffcult to establsh practcal and precse mathematcal models. Data-drven methods provde another way to fnd the causal relatonshps between process varables. A few data-based

2 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2053 methods are capable of detectng the causal relatonshps for lnear processes [6]. In the frequency doman, drected transfer functons (DTFs [7] and partal drected coherence (PDC [8] are wdely used n bran connectvty analyss. Other methods such as Granger causalty [9], path analyss [10], and cross-correlaton analyss wth lag-adjusted varables [11] are commonly used. The predctablty mprovement based on the nearest neghbors s proposed as an asymmetrcal measure of nterdependence n bvarate tme seres and appled to quantfy the drectonal nfluences among physologcal sgnals [12] and also ndustral processes varables [13], [14]. Informaton theory provdes a wde varety of approaches for measurng causal nfluence among multvarate tme seres [15]. Based on transton probabltes contanng all nformaton on causalty between two varables, the transfer entropy (TE was proposed to dstngush between drvng and respondng elements [16] and s sutable for both lnear and nonlnear relatonshps; t has been successfully used n chemcal processes [17] and neuroscences [18]. TE has two forms, dscrete TE (TE dsc for dscrete random varables [16], and dfferental TE (TE dff for contnuous random varables [19]. It has been shown n [20] that, for Gaussan dstrbuted varables wth lnear relatonshps, Granger causalty and TE are equvalent. The equvalence of the two causalty measures has been extended under certan condtons on probablty densty dstrbutons of the data [21]. In [22], comparsons are gven for several causalty detecton methods; these methods nclude TE, extended and nonlnear Granger causalty, and predctablty mprovement. That paper also ncludes a dscusson on the usefulness of the methods for detectng asymmetrc couplngs and nformaton flow drectons n the determnstc chaotc systems. The authors conclude that, gven a complex system wth aprorunknown dynamcs, the frst method of choce mght be TE. If a large number of samples are avalable, the alternatve methods mght be nonlnear Granger causalty and predctablty mprovement. Snce nformaton flow specfcally means how varaton propagates from one varable to another, t s valuable to detect whether the causal nfluence between a par of varables s along a drect pathway wthout any ntermedate varables or ndrect pathways through some ntermedate varables. In the frequency doman, a DTF/PDC-based method for quantfcaton of drect and ndrect energy flow n a multvarate process was recently proposed [23]. Ths method was based on vector auto-regressve or vector movng-average model representatons, whch are sutable for lnear multvarate processes. In the tme doman, a path analyss method was used to calculate the drect effect coeffcents [24]. The calculaton was based on a regresson model of the varables, whch captures only lnear relatonshps. For both lnear and nonlnear relatonshps, based on a multvarate verson of TE, the partal TE was proposed to quantfy the total amount of ndrect couplng medated by the envronment and was successfully used n neuroscences [25]. In [25]. The partal TE s defned such that all the envronmental varables are consdered as ntermedate varables, whch s not necessary n most cases; and n any case, ths wll ncrease the computatonal burden sgnfcantly. On the other hand, the utlty of the partal TE s to detect undrectonal causaltes, whch s sutable for neuroscences; however, n ndustral processes, feedback and bdrectonal causaltes, due to recycle streams, are common. Thus, the partal TE method cannot be drectly used for drect/ndrect causalty detecton n the process ndustry. The man contrbuton of ths paper s a TE-based methodology to detect and dscrmnate between drect and ndrect causalty relatonshps between process varables of both lnear and nonlnear multvarate systems. Specfcally, ths method s able to uncover explct drect and ndrect, as f through ntermedate varables, connectvty pathways between varables. The rest of ths paper s organzed as follows. In Secton II, we apply the TE dff for contnuous random varables to detect total causalty and defne a dfferental drect transfer entropy (DTE dff 1 to detect drect causalty. Calculaton methods and the normalzaton methods are proposed for both the TE dff and the DTE dff n the same secton. Secton III descrbes three examples to show the effectveness of the proposed drect causalty detecton method. An expermental case study and an ndustral case study are ntroduced n Secton IV to show the usefulness of the proposed method for detectng drect/ndrect connectng pathways, followed by concludng remarks n Secton V. II. DETECTION OF DIRECT CAUSALITY In ths secton, an extenson of the TE drect transfer entropy (DTE s proposed to detect the drect causalty between two varables. In addton to ths, calculaton methods and the normalzaton methods are also presented for the TE and the DTE, respectvely. A. DTE In order to determne the nformaton and materal flow pathways to construct a precse topology of a process, t s mportant to determne whether the nfluence between a par of process varables s along drect or ndrect pathways. The drect pathway means drect nfluence wthout any ntermedate or confoundng varables. The TE measures the amount of nformaton transferred from one varable x to another varable y. Ths extracted transfer nformaton represents the total causal nfluence from x to y. It s dffcult to dstngush whether ths nfluence s along a drect pathway or ndrect pathways through some ntermedate varables. For example, gven three varables x, y, andz, f the calculated transfer entropes from x to y, from x to z, and from z to y are all larger than zero, then we can conclude that x causes y, x causes z, andz causes y. We can also conclude that there s an ndrect pathway from x to y va the ntermedate varable z whch transfers nformaton from x to y. However, we cannot dstngush whether there s a drect pathway from x to y (see Fg. 1, because t s possble that there exst both a drect pathway from x to y and an ndrect pathway va the ntermedate varable z. 1 We cauton the reader to be aware of the term DTE dff for dfferental drect transfer entropy and that t s dfferent from the term dscrete drect transfer entropy (DTE dsc as t apples to dscrete random varables.

3 2054 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 x Fg. 1. Detecton of drect causalty from x to y. z? In order to detect the drect and ndrect pathways of the nformaton transfer, the defnton of a DTE s ntroduced as follows. Snce the data analyzed here s unformly sampled data, as obtaned from processes that are contnuous, we only consder contnuous random varables n ths paper. Gven three contnuous random varables x, y, andz, let them be sampled at tme nstants and denoted by x [x mn, x max ], y [y mn, y max ],andz [z mn, z max ] wth = 1, 2,...,N, where N s the number of samples. The causal relatonshps between each par of these varables can be estmated by calculatng the TEs [16]. Let y +h1 denote the value of y at tme nstant + h 1,that s, h 1 steps n the future from, andh 1 s referred to as the predcton horzon; y (k 1 =[y, y τ1,...,y (k1 1τ 1 ] and x (l 1 =[x, x τ1,...,x (l1 1τ 1 ] denote embeddng vectors wth elements from the past values of y and x, respectvely (k 1 s the embeddng dmenson of y and l 1 s the embeddng dmenson of x; τ 1 s the tme nterval that allows the scalng n tme of the embedded vector, whch can be set to be h 1 = τ 1 4 as a rule of thumb [17]; f (y +h1, y (k 1, x (l 1 denotes the jont probablty densty functon (pdf, and f ( denotes the condtonal pdf, and thus f (y +h1 y (k 1, x (l 1 denotes the condtonal pdf of y +h1 gven y (k 1 and x (l 1 and f (y +h1 y (k 1 denotes the condtonal pdf of y +h1 gven. The dfferental TE (TE dff from x to y, for contnuous varables, s then calculated as follows: T x y = f (y +h1, y (k 1, x (l 1 y (k 1 log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k dw (1 1 where the base of the logarthm s 2 and w denotes the random vector [y +h1, y (k 1, x (l 1 ]. By assumng that the elements of w are w 1,w 2,...,w s, ( dw denotes ( dw 1 dw s for smplcty, and the followng notatons have the same meanng as ths one. The TE from x to y can be understood as the mprovement when usng the past nformaton of both x and y to predct the future of y compared to only usng the past nformaton of y. In other words, the TE represents the nformaton about a future observaton of varable y obtaned from the smultaneous observatons of past values of both x and y, after dscardng the nformaton about the future of y obtaned from the past values of y alone. Smlarly, the TE dff from x to z s calculated as follows: T x z = f (z +h2, z (m 1, x (l 2 log f (z +h 2 z (m 1, x (l 2 f (z +h2 z (m 1 y dη (2 where h 2 s the predcton horzon, z (m 1 = [z, z τ2,..., z (m1 1τ 2 ] and x (l 2 =[x, x τ2,...,x (l2 1τ 2 ] are embeddng vectors wth tme nterval τ 2,andη denotes the random vector [z +h2, z (m 1, x (l 2 ]. The TE dff from z to y s calculated as follows: T z y = f (y +h3, y (k 2, z (m 2 log f (y +h 3 y (k 2, z (m 2 f (y +h3 y (k dζ (3 2 where h 3 s the predcton horzon, y (k 2 = [y, y τ3,..., y (k2 1τ 3 ] and z (m 2 = [z, z τ3,...,z (m2 1τ 3 ] are embeddng vectors wth tme nterval τ 3,andζ denotes the random vector [y +h3, y (k 2, z (m 2 ]. If T x y, T x z,andt z y are all larger than zero, then we conclude that x causes y, x causes z, andz causes y. Inths case, we need to dstngush whether the causal nfluence from x to y s only va the ndrect pathway through the ntermedate varable z, or, n addton to ths, there s another drect pathway from x to y, as shown n Fg. 1. We defne a drect causalty from x to y as x drectly causng y, whch means there s a drect nformaton and/or materal flow pathway from x to y wthout any ntermedate varables. In order to detect whether there s a drect causalty from x to y, we defne a dfferental DTE (DTE dff from x to y as follows: D x y = f (y +h, y (k, z (m 2 +h h 3, x (l 1 +h h 1 log f (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 f (y +h y (k, z (m dv (4 2 +h h 3 where v denotes the random vector [y +h, y (k, z (m 2 +h h 3, x (l 1 +h h 1 ]; the predcton horzon h s set to be h = max(h 1, h 3 ;fh = h 1,theny (k = y (k 1,fh = h 3,theny (k = y (k 2 ; the embeddng vector z (m 2 +h h 3 =[z +h h3, z +h h3 τ 3,...,z +h h3 (m 2 1τ 3 ] denotes the past values of z whch can provde useful nformaton for predctng the future y at tme nstant + h, where the embeddng dmenson m 2 and the tme nterval τ 3 are determned by (3; the embeddng vector x (l 1 +h h 1 = [x +h h1, x +h h1 τ 1,...,x +h h1 (l 1 1τ 1 ] denotes the past values of x whch can provde useful nformaton to predct the future y at tme nstant +h, where the embeddng dmenson l 1 and the tme nterval τ 1 are determned by (1. Note that the parameters n DTE dff are all determned by the calculaton of the TEs for consstency. The DTE dff represents the nformaton about a future observaton of y obtaned from the smultaneous observaton of past values of both x and z, after dscardng the nformaton about the future y obtaned from the past z alone. Ths can mean that f the pathway from z to y s cut off, wll the hstory of x stll provde some helpful nformaton to predct the future y? Obvously, f ths nformaton s nonzero (greater than zero, then there s a drect pathway from x to y. Otherwse,theres no drect pathway from x to y, and the causal nfluence from x to y s all along the ndrect pathway va the ntermedate varable z.

4 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2055 Note that the drect causalty here s a relatve concept; snce the measured process varables are lmted, the drect causalty analyss s only based on these varables. In other words, even f there are ntermedate varables n the connectng pathway between two measured varables, as long as none of these ntermedate varables s measured, we stll state that the causalty s drect between the par of measured varables. After the calculaton of D x y, f there s drect causalty from x to y, we need to further judge whether the causalty from z to y s true or spurous, because t s possble that z s not a cause of y and the spurous causalty from z to y s generated by x,.e., x s the common source of both z and y. As shown n Fg. 2, there are stll two cases of the nformaton flow pathways between x, y, andz, andthe dfference s whether there s true and drect causalty from z to y. Thus, DTE dff from z to y needs to be calculated D z y = p(y +h, y (k, x (l 1 +h h 1, z (m 2 +h h 3 log p(y +h y (k, x (l 1 +h h 1, z (m 2 +h h 3 p(y +h y (k, x (l dv (5 1 +h h 1 where the parameters are the same as n (4. If d z y > 0, then there s true and drect causalty from z to y, asshown n Fg. 2(a. Otherwse, the causalty from z to y s spurous, whch s generated by the common source x, asshownn Fg. 2(b. The need for detecton of drect and ndrect causalty based on measured process varables s dscussed. The tradtonal TE method only determnes whether there s causalty from x to y, but we cannot tell whether the causal nfluence s along a drect pathway or ndrect pathways through some ntermedate varables (see Fg. 1. The purpose of process causalty analyss s to nvestgate propagaton of faults, alarms events, and sgnals through materal and nformaton flow pathways (for example va feedback control and n ths respect t s mportant to know whether the connecton between varables of nterest s drect or ndrect. As shown n Fg. 1, f drect causalty from x to y s detected, then there should be a drect nformaton and/or materal flow pathway from x to y. Otherwse, there s no drect nformaton and/or materal flow pathway from x to y and the drect lnk should be elmnated. Ths s clearly llustrated n the expermental three-tank case study presented n Secton IV. Such cases are common n ndustral processes; the tradtonal TE approach wll reveal a myrad of connectons, as t s not able to dscrmnate between drect and ndrect causalty, whereas once one s able to detect drect paths, the number of connectng pathways reduces sgnfcantly. Another mportant case s to detect the true or spurous causalty as shown n Fg. 2. In fact, ths can tell whether there s a drect nformaton and/or materal flow pathway from z to y or there s no nformaton flow pathway from z to y at all. If we only use the tradtonal TE method, we may conclude that there s causal nfluence from z to y and therefore there s an nformaton flow pathway from z to y, whch s not true because they are both nfluenced by a common cause. Thus, (a Fg. 2. Informaton flow pathways between x, y, andz wth (a true and drect causalty from z to y, and (b spurous causalty from z to y (meanng that z and y have a common perturbng source, x, and therefore they may appear to be connected or correlated even when they are not connected physcally. the detecton of drect and ndrect causalty s necessary for capturng the true process connectvty. An mportant applcaton of causalty analyss for capturng process connectvty s to fnd the fault propagaton pathways and dagnose the root cause of certan dsturbance or faults. If we only detect causalty va the tradtonal TE approach, total causalty and spurous causalty would be detected to yeld an overly complcated set of pathways from whch root cause dagnoss of faults would be dffcult f not erroneous. However, f we are able to dfferentate between drect and ndrect, true and spurous causalty, then the derved causal map may be much smpler and more accurate to tell the fault propagaton pathways and whch varable s the lkely root cause. Ths pont s clearly llustrated by the ndustral case study presented n Secton IV. B. Relatonshps Between DTE dff and DTE dsc The TE dff and the DTE dff mentoned above are defned for contnuous random varables. For contnuous random varables, a wdely used TE calculaton procedure s to perform quantzaton frst and then use the formula of TE dsc [17]. Thus, we need to establsh a connecton between ths quantzatonbased procedure and the TE dff procedure. For the contnuous random varables x, y,andz,let x, ỹ and z denote the quantzed x, y, andz, respectvely. Assume that the supports of x, y, andz,.e.,[x mn, x max ], [y mn, y max ],and [z mn, z max ], are classfed nto n x, n y,andn z nonoverlappng ntervals (bns, respectvely, and the correspondng quantzaton bn szes of x, y, andz are x, y,and z, respectvely. Takng x for an example, f we choose a unform quantzer, then we have x = x max x mn. n x 1 We can see that the quantzaton bn sze s related to the varable support and the number of quantzaton ntervals (bn number. Gven a varable support, the larger the bn number, the smaller s the quantzaton bn sze. After quantzaton, the TE from x to y can be approxmated by the TE dsc from x to ỹ t x ỹ = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h 1 ỹ (k 1, x (l 1 p(ỹ +h1 ỹ (k (6 1 where the sum symbol represents k 1 + l 1 + 1sumsoverall ampltude bns of the jont probablty dstrbuton and condtonal probabltes; ỹ (k 1 =[ỹ, ỹ τ1,...,ỹ (k1 1τ 1 ] and =[ x, x τ1,..., x (l1 1τ 1 ] denote embeddng vectors; x (l 1 (b

5 2056 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 p(ỹ +h1, ỹ (k 1, x (l 1 denotes the jont probablty dstrbuton; and p( denotes the condtonal probabltes. The meanng of other parameters remans unchanged. From (6 we can express TE dsc usng condtonal Shannon entropes [26] by expandng the logarthm, as t x ỹ = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h 1, ỹ (k 1, x (l 1 p(ỹ (k 1, x (l 1 p(ỹ +h1, ỹ (k 1 log p(ỹ +h 1, ỹ (k 1 p(ỹ (k 1 = H (ỹ +h1 ỹ (k 1 H (ỹ +h1 ỹ (k 1, x (l 1 (7 where H (ỹ +h1 ỹ (k 1 = p(ỹ +h1, ỹ (k 1 log p(ỹ +h1 ỹ (k 1 and H (ỹ +h1 ỹ (k 1, x (l 1 = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h1 ỹ (k 1, x (l 1 are the condtonal Shannon entropes. Smlar to TE dsc, we can express the TE dff usng dfferental condtonal entropes, as T x y = f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw f (y +h1, y (k 1 log f (y +h1 y (k 1 du = H c (y +h1 y (k 1 H c (y +h1 y (k 1, x (l 1 (8 where u denotes the random vector [y +h1, y (k 1 ], and H c (y +h1 y (k 1 and H c (y +h1 y (k 1, x (l 1 are the dfferental condtonal entropes. Theoretcally, as the bn szes approach zero, the probablty p(ỹ +h1, ỹ (k 1, x (l 1 n (7 can be approxmated by y k 1 y l 1 x f (y +h1, y (k 1, x (l 1. Thenwehave lm x, y 0 t x ỹ = lm x, y 0 { y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 log y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 k 1 y l 1 x f (y (k 1, x (l 1 y k 1 y f (y +h1, y (k 1 log y k 1 y f (y +h1, y (k 1 } k 1 y f (y (k 1 { = lm y k 1 y x, y 0 l 1 x f (y +h1, y (k 1, x (l 1 ( log y + log f (y +h1 y (k 1, x (l 1 y k 1 y f (y +h1, y (k 1 ( log y + log f (y +h1 y (k 1 }. (9 As x, y 0, we have y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 f (y +h1, y (k 1, x (l 1 dw = 1, y k 1 y f (y +h1, y (k 1 f (y +h1, y (k 1 du = 1 and the ntegral of the functon f ( log f ( can be approxmated n the Remannan sense by y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw y k 1 y f (y +h1, y (k 1 log f (y +h1 y (k 1 f (y +h1, y (k 1 log f (y +h1 y (k 1 du. Thus lm t x ỹ x, y 0 = lm log y y 0 + f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw lm log y y 0 f (y +h1, y (k 1 log f (y +h1 y (k 1 du = f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw f (y +h1, y (k 1 log f (y +h1 y (k 1 du = f (y +h1, y (k 1, x (l 1 log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k dw 1 = T x y. (10 Ths means that the dfferental TE from x to y s the same as the dscrete TE from quantzed x to quantzed y n the lmt as the quantzaton bn szes of both x and y approach zero. Remark: From (9 and (10, we can see that the dfference between the dfferental condtonal entropy and the lmtng value of the Shannon condtonal entropy as x, y 0 s an nfnte offset, lm y 0 log y. Thus, the dfferental condtonal entropy can be negatve. Smlar to TE, the DTE from x to y can be approxmated by a dscrete DTE (DTE dsc from x to ỹ d x ỹ = p(ỹ +h, ỹ (k, z (m 2 +h h 3, x (l 1 +h h 1 log p(ỹ +h ỹ (k, z (m 2 +h h 3, x (l 1 +h h 1 p(ỹ +h ỹ (k, z (m 2 +h h 3 (11 where ỹ (k, z (m 2 +h h 3,and x (l 1 +h h 1 are embeddng vectors of ỹ, z, and x, respectvely. The defntons of the other quanttes are smlar to that n (4.

6 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2057 X j, j = 1, 2,...,m, and μ = m j=1 μ j /m; then the standard error of the estmated mean μ s gven by mj=1 (μ j μ 2 σ = m(m 1 Fg. 3. Relatonshps between TEs and DTEs. RV means random varable. For the DTE dff and the DTE dsc, usng the same proof procedure wth the TE, we can obtan lm x, y, z 0 d x ỹ = D x y whch means that the DTE dff from x to y s the same as the DTE dsc from quantzed x to quantzed y n the lmt as the quantzaton bn szes of x, y, and the ntermedate varable z approach zero. Fg. 3 llustrates the relatonshps between TE dff and TE dsc, and between DTE dff and DTE dsc. It should be noted that the smaller the bn sze, the more accurate s the quantzaton and the closer are DTE dsc and DTE dff. Note that the computatonal burden of the summaton and the probablty estmaton n (6 and (11 wll ncrease sgnfcantly wth ncreasng quantzaton bn numbers,.e., n x, n y,andn z. Thus, for the choce of bn szes, there s a tradeoff between the quantzaton accuracy and the computatonal burden n TE dsc and DTE dsc calculatons. In practce, the condtons that the quantzaton bn szes approach zero are dffcult to satsfy. Thus, n order to avod the roundoff error of quantzaton, we drectly use TE dff and DTE dff to calculate TE and DTE, respectvely. C. Calculaton Method 1 Requred Assumptons for the DTE Calculaton: Snce the concept of DTE s an extenson of TE, the requred assumptons for DTE s exactly the same as TE; the collected sampled data must be statonary n a wde sense wth a large data length, preferably no less than 2000 observatons [17]. Statonarty requres that the dynamcal propertes of the system must not change durng the observaton perod. Snce n most cases we do not have drect access to the system and we cannot establsh evdence that ts parameters are ndeed constant, we have to test for statonarty based on the avalable dataset. For the purpose of testng for statonarty, the smplest and most wdely used method s to measure the mean and the varance for several segments of the dataset (equvalent to an ergodcty test and then use a standard statstcal hypothess test to check whether the mean and the varance change. More subtle quanttes such as spectral components, correlatons, or nonlnear statstcs may be needed to detect less obvous nonstatonarty [27]. In ths paper, we use the mean and varance to test for statonarty. We dvde a gven dataset, denoted by x, = 1, 2,...,N, nto m consecutve segments, denoted by X 1, X 2,...,X m, each contanng s data ponts. Let μ j denote the mean value of where the standard devaton dvded by an extra m s the error when estmatng the mean value of Gaussan dstrbuted uncorrelated numbers [27]. The null hypothess for statonarty testng s that the dataset s statonary. The sgnfcance level for the mean testng s defned as μ j μ > 6 for j = 1, 2,...,m. (12 σ A sx-sgma threshold for the sgnfcance level s chosen here. Specfcally, f there exsts μ j > μ + 6σ or μ j < μ 6σ for j = 1, 2,...,m, then the null hypothess that the dataset s statonary s rejected. If μ 6σ < μ j < μ + 6σ holds for all js, then the null hypothess s accepted that the dataset s statonary. For the varance test, let ˆx, = 1, 2,...,N denote the normalzed dataset of x, and x 1, x 2,..., x m denote the correspondng consecutve segments. Then we have x j = ˆx s( j 1+1, ˆx s( j 1+2,..., ˆx sj for j = 1, 2,...,m. Snce the sum of squares of the elements n each segment has the ch-squared dstrbuton wth s degrees of freedom ˆv j = ˆx s( 2 j 1+1 +ˆx s( 2 j ˆx sj 2 χ s 2, we can check whether or not the dataset s statonary by comparng ˆv j wth χs 2(α. If there exsts ˆv j > χs 2 (α for j = 1, 2,...,m, then the null hypothess that the dataset s statonary s rejected wth (1 α 100% confdence. If ˆv j <χs 2 (α for all js, then the null hypothess s accepted. Multmodalty s often encountered n ndustral processes due to the normal operatonal changes as well as changes n the producton strategy [28]. For such multmodal processes, a dataset wth a large number of samples s most lkely to be nonstatonary as the data would reflect transtons from one mode to another, whereas a key assumpton of the TE/DTE method s statonarty of the sampled data. In order to handle the process multmodalty, one would have to partton the data nto dfferent segments correspondng to dfferent modes. A few tme-seres analyss methods [29], [30] have been proposed for segmentaton of tme seres to determne when the process mode has changed. As long as the segments correspondng to dfferent modes are obtaned, we can detect (drect causalty for each mode of the process usng the approprate segment. Note that the causal relatonshps may change wth mode swtchng of the process. 2 Estmaton of the TE dff and the DTE dff : For the TE from x to y, snce (1 can be wrtten as { T x y = E t can be approxmated by log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k 1 N h 1 1 T x y = log f (y +h 1 y (k 1, x (l 1 N h 1 r + 1 =r f (y +h1 y (k 1 } (13

7 2058 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 where N s the number of samples and r = max{(k 1 1 τ 1 + 1,(l 1 1τ 1 + 1}. Just as wth TE dff,thedte dff (4 can be wrtten as {log f (y +h y (k, z (m 2 +h h 3, x (l } 1 +h h 1 D x y = E whch can be approxmated by D x y = 1 N h j + 1 N h = j f (y +h y (k, z (m 2 +h h 3 log f (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 f (y +h y (k, z (m 2 +h h 3 (14 where j = max{(k 1 1τ 1 + 1,(k 2 1τ 3 + 1, h + h 3 + (m 2 1τ 3 + 1, h + h 1 + (l 1 1τ 1 + 1}. 3 Kernel Estmaton of pdfs: In (13 and (14, the condtonal pdfs are expressed by the jont pdfs and then obtaned by the kernel estmaton method [31]. Here, the followng Gaussan kernel functon s used k(u = 1 2π e 1 2 u2. Then a unvarate pdf can be estmated by f ˆ(x = 1 N ( x X k Nγ γ =1 (15 where N s the number of samples, and γ s the bandwdth chosen to mnmze the mean ntegrated squared error of the pdf estmaton and calculated by γ = 1.06σ N 1/5 accordng to the normal reference rule-of-thumb [31], [32], where σ s the standard devaton of the sampled data {X } =1 N. For q-dmensonal multvarate data, we use the Fukunaga method [31] to estmate the jont pdf. Suppose that X 1,...,X N consttute a q-dmensonal vector (X R q wth a common pdf f (x 1, x 2,...,x q. Let x denote the q-dmensonal vector [x 1, x 2,...,x q ] T ; then the kernel estmaton of the jont pdf s f ˆ (det S 1/2 (x = NƔ q N K =1 { } Ɣ 2 (x X T S 1 (x X (16 where Ɣ s smlar to the bandwdth γ n (15. The estmated jontpdfs smootherwhenɣ s larger. However, a substantally larger Ɣ s most lkely to result n an naccurate estmaton. Thus, Ɣ s also chosen to mnmze the mean ntegrated squared error of the jont pdf estmaton and calculated by Ɣ = 1.06N 1/(4+q. S s the covarance matrx of the sampled data, and K s the Gaussan kernel satsfyng K (u = (2π q/2 e 1 2 u. Note that when q = 1, (16 s smplfed nto (15. For the TE, the estmaton of the computatonal complexty s dvded nto two parts: the kernel estmaton of the pdf usng (16, and the calculaton of the TE dff usng (13. For each jont pdf of dmenson q, the computatonal complexty s O(N 2 q 2. Consderng the condtonal pdfs are estmated by the jont pdfs, the maxmum dmenson of the jont pdf s k 1 + l and, thus, the computatonal complexty for the pdf estmaton s O(N 2 (k 1 + l 1 2. For calculaton of the TE dff n (13, approxmately N summatons are requred. Thus, the total computatonal complexty for the TE dff s O(N 2 (k 1 + l 1 2. Smlarly, we can obtan that the computatonal complexty for the DTE dff usng (14 s O(N 2 (k + m 2 + l 1 2. It s obvous that the number of samples and the embeddng dmensons determne the computng speed. Snce the samples number s preferred to be no less than 2000 observatons [17], we need to lmt the choce of the embeddng dmensons. The detals on how to choose the embeddng dmensons are gven n the followng subsecton. Note that the computatonal complexty s relatvely large because of the kernel estmaton of the (jont pdfs, and that the computatonal complexty for the DTE ncreases wth an ncreasng number of ntermedate varables. Therefore, one would have to apply the method to smaller unts wth a smaller number of varables. A large-scale complex system can be broken down nto smaller unts and thereafter analyzed for causal relatonshps wthn each unt and between dfferent unts, and fnally the nformaton flow pathways of the whole process can be establshed. 4 Determnaton of the Parameters of the TE: In the use of the TE approach to detect causalty, there are four undetermned parameters: 1 the predcton horzon (h 1 ; 2 the tme nterval (τ 1 ; 3 and the embeddng dmensons (k 1 and l 1. Snce these four parameters greatly affect the calculaton results of the transfer entropes, we need to fnd a systematc method to determne them. Frst, snce h 1 = τ 1 4 as a rule of thumb [17], we can further set ntal values for h 1 and τ 1 accordng to apror knowledge of the process. For example, we start by settng the ntal values for h 1 = τ 1 = 1. Second, we can determne the embeddng dmenson of y,.e., the wndow sze of the hstorcal y used for the future y predcton. The embeddng dmenson of y,.e., k 1, can be determned as the mnmum nonnegatve nteger above whch the change rate of H c (y +h1 y (k 1 decreases sgnfcantly. Consderng a large k 1 can ncrease the dmenson of the jont pdf and the dffculty n pdf estmaton, f k 1 s greater than 3, we need to ncrease h 1 and τ 1 and repeat the calculaton untl a k 1 3 s found to make the change rate of H c (y +h1 y (k 1 decrease sgnfcantly. Fnally, we can determne the embeddng dmenson of x,.e., the wndow sze of the hstorcal x used for the future y predcton. Based on the values of k 1, h 1,, and τ 1, the embeddng dmenson of x,.e., l 1, s determned as the mnmum postve nteger above whch the change rate of the TE from x to y decreases sgnfcantly. 5 Normalzaton: It s easy to prove that both the TE and the DTE are condtonal mutual nformaton; thus they are always nonnegatve. However, small values of the TE and the DTE suggest no causalty or drect causalty whle large values do. In order to quantfy the strength of the total causalty and drect causalty, normalzaton s necessary.

8 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2059 In [33], the normalzed dscrete TE (NTE dsc sdefnedas NTE x ỹ = t x ỹ t shuffled x ỹ H (ỹ +h1 ỹ (k [0, 1] (17 1 where t shuffled x ỹ s an estmate of the same TE n shuffled data of x and ỹ. ThsNTE dsc ntutvely represents the fracton of nformaton n ỹ not explaned by ts own past but explaned by the past of x. Eq. (17 s sutable for the normalzaton of the TE dsc. For TE dff, we cannot just substtute H (ỹ +h1 ỹ (k 1 wth the dfferental condtonal entropy H c (y +h1 y (k 1, snce H c (y +h1 y (k 1 could be negatve. Moreover, usng shuffled data to elmnate the calculaton bas s not accurate because random shufflng may destroy the statstcal propertes of the tme seres. Also, t shuffled x ỹ s an average of transfer entropes obtaned on n trals. To obtan a better result, n should be large enough, whch wll ncrease the computatonal burden sgnfcantly. Thus, we need to propose a new normalzaton method for TE dff. In (17 the zero pont s regarded as the orgn and t represents a determnstc varable. For dfferental entropy, the value nstead of zero means that the varable s determnstc. The maxmal dfferental entropy gven a fnte support s n the form of a unform dstrbuton [34]. So, we defne the orgn as the maxmal dfferental entropy of y wth the unform dstrbuton ymax 1 H 0 (y = log dy y max y mn y max y mn = log(y max y mn y mn 1 where y max and y mn denote the maxmum and mnmum values of the varable y, respectvely. Consderng that the TE dff s the dfference between two dfferental condtonal entropes, as shown n (8, we defne the normalzed dfferental TE (NTE dff as NTE c x y = H c (y +h1 y (k 1 H c (y +h1 y (k 1, x (l 1 H 0 H c (y +h1 y (k 1, x (l 1 T x y = H 0 H c (y +h1 y (k 1, x (l 1 [0, 1]. (18 Intutvely, the numerator term represents the TE to capture the nformaton about y not explaned by ts own hstory and yet explaned by the hstory of x; the denomnator term represents the nformaton n y that s provded by the past values of both x and y. It s obvous that NTE c x y = 0f T x y = 0. If y s unformly dstrbuted and the nformaton about y explaned by the hstory of both x and y s completely explaned by the hstory of x, whch means H c (y +h1 y (k 1 = H 0, then accordng to (18 we obtan NTE c x y = 1. Snce an entropy H represents the average number of bts needed to optmally encode ndependent draws of a random varable [16], the uncertan nformaton contaned n a sgnal s n fact proportonal to 2 H. Here, a sgnal means a specfc realzaton of the random varable. We extend the lnear normalzaton functon n (18 to a nonlnear functon as follows: NTE c x y = 2H c (k (y +h1 y 1 2 H c (y +h1 y (k 1,x (l 1 2 H 0 2 H c (y +h1 y (k 1,x (l 1 [0, 1]. (19 The meanng of (19 s the same as that n (18. Ths nonlnear normalzaton functon (19 wll be used later. Snce the DTE dff n (4 represents the nformaton drectly provded from the past x to the future y, a normalzed dfferental DTE (NDTE dff sdefnedas D x y NDTE c x y = H c (y +h y (k H c (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 [0, 1 (20 where H c (y +h y (k and H c (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 are the dfferental condtonal entropes. Intutvely, ths NDTE dff represents the percentage of drect causalty from x to y n the total causalty from both x and z to y. D. Extenson to Multple Intermedate Varables The defnton of the DTE dff from x to y can be easly extended to multple ntermedate varables z 1, z 2,...,z q,as D x y = f (y +h, y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 +h h 1 log f (y +h y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 f (y +h y (k, z (s 1 1, 1,...,z (s q q, q +h h 1 dξ (21 where s 1,...,s q and 1,..., q are the correspondng parameters determned by the calculatons of the transfer entropes from z 1,...,z q to y, and ξ denotes the random vector [y +h, y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 +h h 1 ]. If d x y s zero, then there s no drect causalty from x to y, and the causal effects from x to y are all along the ndrect pathways va the ntermedate varables z 1, z 2,...,z q.ifd x y s larger than zero, then there s drect causalty from x to y. The formulatons of the proposed DTE and the partal TE n [25] are smlar, but the basc deas are qute dfferent. The major dfference s that for the partal TE, all the envronmental varables are consdered as ntermedate varables, whereas for the DTE, the ntermedate varables are chosen based on calculaton results from the tradtonal TE. Specfcally, the partal TE was proposed as a substtuton of the tradtonal TE. We can choose ether tradtonal TE to detect total causalty or partal TE to detect partal causalty only. However, the DTE was proposed here as an extenson of the tradtonal TE and should be used after capturng the nformaton flow pathways va the tradtonal TE method. The ntermedate varables from x to y are determned as the varables wthn the nformaton flow pathway from x to y (see Fg. 1 and common sources of both x and y (see Fg. 2. More specfc comparsons between the partal TE and the DTE are as follows:

9 2060 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER The partal TE s defned such that all the envronmental varables are consdered as ntermedate varables, whch s not necessary n most cases and n any case ths wll ncrease the computatonal burden sgnfcantly. Moreover, ths may even result n false causalty detecton. For example, gven three varables x, y, andz, assume that x and y are ndependent and the true causal relatonshp between them s that x causes z and y also causes z,.e., x z y. It s clear that, gven the nformaton of z, x and y are no longer ndependent. If we use the partal TE to detect the causalty from x to y, gven nformaton of z, t s most lkely to conclude that there s causalty between x and y as long as there s a tme delay between them, although n fact the detected causalty between x and y does not exst. Whle usng the DTE method, we need to frst detect causalty by usng the tradtonal TE method, and then determne whch varable s the ntermedate varable. Snce the nformaton of z s unknown whle usng the tradtonal TE method n whch only two varables x and y are consdered, the causalty between x and y cannot be detected, and then we conclude that there s no causal relatonshp between x and y, whch s consstent wth the fact. 2 For calculatons of the tradtonal TE, the partal TE, and the DTE, t s mportant to determne the parameters for each varable,.e., the predcton horzon, embeddng dmensons, and the tme nterval. Much research has been done on how to determne these parameters for the tradtonal TE. For example, n [17], many smulatons have been done to determne these parameters. In ths paper, we also propose a parameter determnaton method for calculatng the tradtonal TE n Secton II- C. We can see that, for only two varables, t s not easy to determne the parameters. If all the envronmental varables are consdered as ntermedate varables, the parameters for a large number of varables need to be chosen smultaneously and approprately, whch s nontrval to acheve. But, for the DTE method, the parameters are determned based on the calculaton of the tradtonal transfer entropes; thus, as long as the parameters of the tradtonal TE for each par of the varables are determned, the parameters n the formulaton of the DTE can be determned accordngly. Detals can be found n the defnton of the DTE n Secton II-A. 3 From the applcaton pont of vew, the utlty of the partal TE s to detect undrectonal causaltes [25]. The authors of [25] use the dfference between the partal TE from x to y and the partal TE from y to x to quantfy the causalty from x to y, whch s sutable n neuroscences; however, n ndustral processes, feedback and bdrectonal causaltes are common due to recycle streams. If we stll use the dfference between TE from x to y and the TE from y to x to quantfy the causalty from x to y, t s most lkely to lead to the concluson that there s no causal relatonshp between x and y, although n fact there s bdrectonal causalty between x and y. Thus, we use the calculated TE and DTE to quantfy total causalty and drect causalty, respectvely. In addton, we propose normalzaton methods to quantfy the strength of the total causalty and drect causalty, respectvely. III. EXAMPLES In ths secton, we gve three examples to show the usefulness of the proposed method. The frst two examples use smple mathematcal equatons to represent causal relatonshps and the thrd example s a smulated 2 2 multple-nput multple-output (MIMO system. Example 1: Assume three lnear correlated contnuous random varables x, y, andz satsfyng { zk+1 = 0.8x k + 0.2z k + v 1k y k+1 = 0.6z k + v 2k where x k N(0, 1, v 1k,v 2k N(0, 0.1, andz(0 = 3.2. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. To calculate the transfer entropes between x, z, and y, we need to determne the four desgn parameters. We take the TE from x to y n (1 as an example. Frst, we set ntal values for h 1 and τ 1 as h 1 = τ 1 = 1. Second, we calculate H c (y +h1 y (k 1 wth k 1 = 0, 1,...,10, as shown n the upper part of Fg. 4. The change rate of H c (y +h1 y (k 1 wth k 1 = 0, 1,...,10 s shown n the lower part of Fg. 4; we can see that as k 1 ncreases, the change rate of H c (y +h1 y (k 1 does not vary sharply, whch means that the hstory of y does not provde useful nformaton for the future values of y. Therefore, we choose k 1 = 0. Fnally, we calculate the TE T x y and ts change rate wth l 1 = 1,...,10, as shown n Fg. 5. Snce the change rate of T x y decreases sgnfcantly after l 1 = 2, as shown n the lower part of Fg. 5, we choose l 1 = 2. Usng the same procedure, the parameters for each par of x, z, andy are determned as h 1 = h 2 = h 3 = 1, τ 1 = τ 2 = τ 3 = 1, k 1 = m 1 = k 2 = 0, l 1 = 2, and l 2 = m 2 = 1. For the followng example and case studes, the same procedure s used. After the parameters are determned accordng to (19, the normalzed transfer entropes between each par of x, y, andz are shown n Table I. We can see that x causes z, z causes y, and x causes y because NTE c x z = 0.409, NTEc z y = 0.393, and NTE c x y = are relatvely large. Thus we need to frst determne whether there s drect causalty from x to y. Accordng to (4, we obtan D x y = Accordng to (20, the normalzed DTE from x to y s NDTE c x y = 0.016, whch s very small. Thus, we conclude that there s almost no drect causalty from x to y. The nformaton flow pathways for Example 1 are shown n Fg. 6(a. Ths concluson s consstent wth the mathematcal functon, from whch we can see that the nformaton flow from x to y s through the ntermedate varable z and there s no drect nformaton flow pathway from x to y. Example 2: Assume three nonlnear correlated contnuous random varables x, y, andz satsfyng { zk+1 = (0.8x k z k +v 1k y k+1 = 5(z k x k +v 2k

10 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE y( k 1 y H ( c Fg. 6. Informaton flow pathways for (a Example 1. (b Example 2. 0 k 1 TABLE II NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 2 +1 y( k 1 ΔH (y c k 1 NTE c row column x z y x NA z 0 NA y NA Fg. 4. Fndng the embeddng dmenson of y for Example T x y l 1 Fg. 7. System block dagram for Example 3. Δ T x y l 1 Fg. 5. Fndng the embeddng dmenson of x for T x y of Example 1. TABLE I NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 1 NTE c row column x z y x NA z NA y NA where x k [4, 5] s a unform dstrbuted sgnal, v 1k,v 2k N(0, 0.05, and z(0 = 0.2. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. The normalzed transfer entropes between each par of x, z, andy are shown n Table II. We can see that x causes z, z causes y, andx causes y because NTE c x z = 0.623, NTE c z y = 0.308, and NTE c x y = are relatvely large. Thus, we need to frst determne whether there s drect causalty from x to y. Accordng to (4, we obtan D x y = Accordng to (20, the normalzed DTE from x to y s NDTE c x y = 0.304, whch s much larger than zero. Thus, we conclude that there s drect causalty from x to y. Second, we need to detect whether there s true and drect causalty from z to y. Accordng to (5, we obtan D z y = 0.538, and thus the normalzed DTE from z to y s NDTE c z y = 0.438, whch s much larger than zero. Hence, we conclude that there s true and drect causalty from z to y. The nformaton flow pathways for Example 2 are shown n Fg. 6(b. Ths concluson s consstent wth the mathematcal functon, from whch we can see that there are drect nformaton flow pathways both from x to y and from z to y. Example 3: Fg. 7 shows a block dagram of a MIMO system wth two nputs r 1 and r 2, and two outputs y 1 and y 2. Assume that r 1 N(0, 1 and r 2 N(0, 1 are ndependent, and v N(0, 0.1 s the sensor nose. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. The normalzed transfer entropes between each par of r 1, r 2, y 1,andy 2 are shown n Table III. We can see that r 1 causes y 1 and y 2, r 2 also cause y 1 and y 2,andy 2 causes y 1. The correspondng nformaton flow pathways are shown n Fg. 8.

11 2062 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 TABLE III NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 3 NTE c row column r 1 r 2 y 1 y 2 r 1 NA r NA y NA y NA Fg. 8. Informaton flow pathways for Example 3. AsshownnFg.8,sncey 1 and y 2 have common sources r 1 and r 2, we need to frst detect whether the causalty from y 2 to y 1 s true or spurous. Accordng to (21, we obtan that the DTE from y 2 to y 1 wth ntermedate varables r 1 and r 2 s D y2 y 1 = Accordng to (20, the normalzed DTE from y 2 to y 1 s NDTE c y 2 y 1 = 0.366, whch s much larger than zero. Hence, we conclude that there s true and drect causalty from y 2 to y 1. Second, snce r 1 causes y 2, y 2 causes y 1,andr 1 causes y 1, we need to further detect whether there s drect causalty from r 1 to y 1. Accordng to (4, we obtan that the DTE from r 1 to y 1 wth the ntermedate varable y 2 s D r1 y 1 = Accordng to (20, the normalzed DTE from r 1 to y 1 s NDTEr c 1 y 1 = 0.573, whch s much larger than zero. Thus, we conclude that there s drect causalty from r 1 to y 1 n addton to the ndrect causalty through ntermedate varable y 2. Smlarly, we obtan that the DTE from r 2 to y 1 wth the ntermedate varable y 2 s D r2 y 1 = and the normalzed DTE from r 2 to y 1 s NDTEr c 2 y 1 = 0.617, whch s also much larger than zero. Thus, we conclude that there s drect causalty from r 2 to y 1. The nformaton flow pathways are the same as those obtaned from the results of calculated TEs, as shown n Fg. 8. Ths concluson s consstent wth the block dagram, from whch we can see that there are drect nformaton flow pathways from r 1 to y 1, from r 2 to y 1, and from y 2 to y 1. No matter whether the relatonshps of varables are lnear or nonlnear, the DTE can detect drect causalty and the normalzed DTE can quantfy the strength of drect causalty. IV. CASE STUDIES In ths secton, an expermental and an ndustral case studes are llustrated to valdate the proposed drect causalty detecton method. A. Expermental Case Study In order to show the effectveness of the proposed methods, a three-tank experment was conducted. The schematc of the three-tank system s shown n Fg. 9. Water s drawn from a Fg. 9. Fg. 10. Schematc of the three-tank system. x 4 x 3 x 2 x 1 Tme Trends Samples Tme trends of measurements of the three-tank system. reservor and pumped to tanks 1 and 2 by a gear pump and a three way valve. The water n tank 2 can flow down nto tank 3. The water n tanks 1 and 3 eventually flows down nto the reservor. The experment s conducted under open-loop condtons. The water levels are measured by level transmtters. We denote the water levels of tanks 1 3 by x 1, x 2, and x 3, respectvely. The flow rate of the water out of the pump s measured by a flow meter; we denote ths flow rate by x 4.In ths experment, the normal flow rate of the water out of the pump s 10 l/mn. However, the flow rate vares randomly wth a mean value of 10 l/mn because of the nose n the sensor and mnor fluctuatons n the pump. The sampled data of 3000 observatons are analyzed. Fg. 10 shows the normalzed tme trends of the measurements. The samplng tme s 1 s. In order to detect the causalty and drect causalty usng TE and DTE, we need to frst test the statonarty of the dataset. Takng x 1 as an example, we dvde the 3000 data ponts nto

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

Quantization Effects in Digital Filters

Quantization Effects in Digital Filters Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value

More information

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

SIMPLE LINEAR CORRELATION

SIMPLE LINEAR CORRELATION SIMPLE LINEAR CORRELATION Smple lnear correlaton s a measure of the degree to whch two varables vary together, or a measure of the ntensty of the assocaton between two varables. Correlaton often s abused.

More information

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

More information

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo.

RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo. ICSV4 Carns Australa 9- July, 007 RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL Yaoq FENG, Hanpng QIU Dynamc Test Laboratory, BISEE Chna Academy of Space Technology (CAST) yaoq.feng@yahoo.com Abstract

More information

Risk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008

Risk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008 Rsk-based Fatgue Estmate of Deep Water Rsers -- Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

Can Auto Liability Insurance Purchases Signal Risk Attitude?

Can Auto Liability Insurance Purchases Signal Risk Attitude? Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159-164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? Chu-Shu L Department of Internatonal Busness, Asa Unversty, Tawan Sheng-Chang

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol

CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL

More information

2.4 Bivariate distributions

2.4 Bivariate distributions page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

Calculation of Sampling Weights

Calculation of Sampling Weights Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample

More information

The Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15

The Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15 The Analyss of Covarance ERSH 830 Keppel and Wckens Chapter 5 Today s Class Intal Consderatons Covarance and Lnear Regresson The Lnear Regresson Equaton TheAnalyss of Covarance Assumptons Underlyng the

More information

1. Measuring association using correlation and regression

1. Measuring association using correlation and regression How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING

ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING Matthew J. Lberatore, Department of Management and Operatons, Vllanova Unversty, Vllanova, PA 19085, 610-519-4390,

More information

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.

More information

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.

Inequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001. Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.

More information

The Analysis of Outliers in Statistical Data

The Analysis of Outliers in Statistical Data THALES Project No. xxxx The Analyss of Outlers n Statstcal Data Research Team Chrysses Caron, Assocate Professor (P.I.) Vaslk Karot, Doctoral canddate Polychrons Economou, Chrstna Perrakou, Postgraduate

More information

ErrorPropagation.nb 1. Error Propagation

ErrorPropagation.nb 1. Error Propagation ErrorPropagaton.nb Error Propagaton Suppose that we make observatons of a quantty x that s subject to random fluctuatons or measurement errors. Our best estmate of the true value for ths quantty s then

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

Damage detection in composite laminates using coin-tap method

Damage detection in composite laminates using coin-tap method Damage detecton n composte lamnates usng con-tap method S.J. Km Korea Aerospace Research Insttute, 45 Eoeun-Dong, Youseong-Gu, 35-333 Daejeon, Republc of Korea yaeln@kar.re.kr 45 The con-tap test has the

More information

Study on CET4 Marks in China s Graded English Teaching

Study on CET4 Marks in China s Graded English Teaching Study on CET4 Marks n Chna s Graded Englsh Teachng CHE We College of Foregn Studes, Shandong Insttute of Busness and Technology, P.R.Chna, 264005 Abstract: Ths paper deploys Logt model, and decomposes

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

An Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services

An Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services An Evaluaton of the Extended Logstc, Smple Logstc, and Gompertz Models for Forecastng Short Lfecycle Products and Servces Charles V. Trappey a,1, Hsn-yng Wu b a Professor (Management Scence), Natonal Chao

More information

CHAPTER 14 MORE ABOUT REGRESSION

CHAPTER 14 MORE ABOUT REGRESSION CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp

More information

The Application of Fractional Brownian Motion in Option Pricing

The Application of Fractional Brownian Motion in Option Pricing Vol. 0, No. (05), pp. 73-8 http://dx.do.org/0.457/jmue.05.0..6 The Applcaton of Fractonal Brownan Moton n Opton Prcng Qng-xn Zhou School of Basc Scence,arbn Unversty of Commerce,arbn zhouqngxn98@6.com

More information

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy S-curve Regresson Cheng-Wu Chen, Morrs H. L. Wang and Tng-Ya Hseh Department of Cvl Engneerng, Natonal Central Unversty,

More information

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP) 6.3 / -- Communcaton Networks II (Görg) SS20 -- www.comnets.un-bremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes

More information

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:

SPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background: SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and

More information

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo

More information

"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *

Research Note APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES * Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC

More information

1 Example 1: Axis-aligned rectangles

1 Example 1: Axis-aligned rectangles COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton

More information

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts

Power-of-Two Policies for Single- Warehouse Multi-Retailer Inventory Systems with Order Frequency Discounts Power-of-wo Polces for Sngle- Warehouse Mult-Retaler Inventory Systems wth Order Frequency Dscounts José A. Ventura Pennsylvana State Unversty (USA) Yale. Herer echnon Israel Insttute of echnology (Israel)

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

Formula of Total Probability, Bayes Rule, and Applications

Formula of Total Probability, Bayes Rule, and Applications 1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.

More information

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6 PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has

More information

Brigid Mullany, Ph.D University of North Carolina, Charlotte

Brigid Mullany, Ph.D University of North Carolina, Charlotte Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte

More information

Ring structure of splines on triangulations

Ring structure of splines on triangulations www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

More information

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6)

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6) Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

Statistical Methods to Develop Rating Models

Statistical Methods to Develop Rating Models Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and

More information

Credit Limit Optimization (CLO) for Credit Cards

Credit Limit Optimization (CLO) for Credit Cards Credt Lmt Optmzaton (CLO) for Credt Cards Vay S. Desa CSCC IX, Ednburgh September 8, 2005 Copyrght 2003, SAS Insttute Inc. All rghts reserved. SAS Propretary Agenda Background Tradtonal approaches to credt

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

Exhaustive Regression. An Exploration of Regression-Based Data Mining Techniques Using Super Computation

Exhaustive Regression. An Exploration of Regression-Based Data Mining Techniques Using Super Computation Exhaustve Regresson An Exploraton of Regresson-Based Data Mnng Technques Usng Super Computaton Antony Daves, Ph.D. Assocate Professor of Economcs Duquesne Unversty Pttsburgh, PA 58 Research Fellow The

More information

Implementation of Deutsch's Algorithm Using Mathcad

Implementation of Deutsch's Algorithm Using Mathcad Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

More information

greatest common divisor

greatest common divisor 4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

On the Optimal Control of a Cascade of Hydro-Electric Power Stations

On the Optimal Control of a Cascade of Hydro-Electric Power Stations On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;

More information

Characterization of Assembly. Variation Analysis Methods. A Thesis. Presented to the. Department of Mechanical Engineering. Brigham Young University

Characterization of Assembly. Variation Analysis Methods. A Thesis. Presented to the. Department of Mechanical Engineering. Brigham Young University Characterzaton of Assembly Varaton Analyss Methods A Thess Presented to the Department of Mechancal Engneerng Brgham Young Unversty In Partal Fulfllment of the Requrements for the Degree Master of Scence

More information

DEFINING %COMPLETE IN MICROSOFT PROJECT

DEFINING %COMPLETE IN MICROSOFT PROJECT CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMI-SP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,

More information

Time Domain simulation of PD Propagation in XLPE Cables Considering Frequency Dependent Parameters

Time Domain simulation of PD Propagation in XLPE Cables Considering Frequency Dependent Parameters Internatonal Journal of Smart Grd and Clean Energy Tme Doman smulaton of PD Propagaton n XLPE Cables Consderng Frequency Dependent Parameters We Zhang a, Jan He b, Ln Tan b, Xuejun Lv b, Hong-Je L a *

More information

Control Charts for Means (Simulation)

Control Charts for Means (Simulation) Chapter 290 Control Charts for Means (Smulaton) Introducton Ths procedure allows you to study the run length dstrbuton of Shewhart (Xbar), Cusum, FIR Cusum, and EWMA process control charts for means usng

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

Data Broadcast on a Multi-System Heterogeneous Overlayed Wireless Network *

Data Broadcast on a Multi-System Heterogeneous Overlayed Wireless Network * JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, 819-840 (2008) Data Broadcast on a Mult-System Heterogeneous Overlayed Wreless Network * Department of Computer Scence Natonal Chao Tung Unversty Hsnchu,

More information

STATISTICAL DATA ANALYSIS IN EXCEL

STATISTICAL DATA ANALYSIS IN EXCEL Mcroarray Center STATISTICAL DATA ANALYSIS IN EXCEL Lecture 6 Some Advanced Topcs Dr. Petr Nazarov 14-01-013 petr.nazarov@crp-sante.lu Statstcal data analyss n Ecel. 6. Some advanced topcs Correcton for

More information

An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Oriented Network Evolution Mechanism for Online Communities An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

More information

L10: Linear discriminants analysis

L10: Linear discriminants analysis L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

More information

IMPACT ANALYSIS OF A CELLULAR PHONE

IMPACT ANALYSIS OF A CELLULAR PHONE 4 th ASA & μeta Internatonal Conference IMPACT AALYSIS OF A CELLULAR PHOE We Lu, 2 Hongy L Bejng FEAonlne Engneerng Co.,Ltd. Bejng, Chna ABSTRACT Drop test smulaton plays an mportant role n nvestgatng

More information

Analysis of Covariance

Analysis of Covariance Chapter 551 Analyss of Covarance Introducton A common tas n research s to compare the averages of two or more populatons (groups). We mght want to compare the ncome level of two regons, the ntrogen content

More information

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account Amercan J. of Engneerng and Appled Scences (): 8-6, 009 ISSN 94-700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a Day-Ahead Electrcty Market wth Rsk Management Taken nto Account

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic

Institute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange

More information

Frequency Selective IQ Phase and IQ Amplitude Imbalance Adjustments for OFDM Direct Conversion Transmitters

Frequency Selective IQ Phase and IQ Amplitude Imbalance Adjustments for OFDM Direct Conversion Transmitters Frequency Selectve IQ Phase and IQ Ampltude Imbalance Adjustments for OFDM Drect Converson ransmtters Edmund Coersmeer, Ernst Zelnsk Noka, Meesmannstrasse 103, 44807 Bochum, Germany edmund.coersmeer@noka.com,

More information

Scalar and Vector Quantization

Scalar and Vector Quantization Scalar and Vector Quantzaton Máro A. T. Fgueredo, Departamento de Engenhara Electrotécnca e de Computadores, Insttuto Superor Técnco, Lsboa, Portugal maro.fgueredo@st.utl.pt November 2008 Quantzaton s

More information

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The

More information

FORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER

FORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 27695-7909 Introducton he convectve heat

More information

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy

Answer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy 4.02 Quz Solutons Fall 2004 Multple-Choce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multple-choce questons. For each queston, only one of the answers s correct.

More information

CHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES

CHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES CHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES In ths chapter, we wll learn how to descrbe the relatonshp between two quanttatve varables. Remember (from Chapter 2) that the terms quanttatve varable

More information

Today s class. Chapter 13. Sources of uncertainty. Decision making with uncertainty

Today s class. Chapter 13. Sources of uncertainty. Decision making with uncertainty Today s class Probablty theory Bayesan nference From the ont dstrbuton Usng ndependence/factorng From sources of evdence Chapter 13 1 2 Sources of uncertanty Uncertan nputs Mssng data Nosy data Uncertan

More information

Nonlinear data mapping by neural networks

Nonlinear data mapping by neural networks Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

More information

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS 21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech-2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS

More information

Lecture 3: Force of Interest, Real Interest Rate, Annuity

Lecture 3: Force of Interest, Real Interest Rate, Annuity Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and

More information

Study on Model of Risks Assessment of Standard Operation in Rural Power Network

Study on Model of Risks Assessment of Standard Operation in Rural Power Network Study on Model of Rsks Assessment of Standard Operaton n Rural Power Network Qngj L 1, Tao Yang 2 1 Qngj L, College of Informaton and Electrcal Engneerng, Shenyang Agrculture Unversty, Shenyang 110866,

More information

Simulation and optimization of supply chains: alternative or complementary approaches?

Simulation and optimization of supply chains: alternative or complementary approaches? Smulaton and optmzaton of supply chans: alternatve or complementary approaches? Chrstan Almeder Margaretha Preusser Rchard F. Hartl Orgnally publshed n: OR Spectrum (2009) 31:95 119 DOI 10.1007/s00291-007-0118-z

More information

New bounds in Balog-Szemerédi-Gowers theorem

New bounds in Balog-Szemerédi-Gowers theorem New bounds n Balog-Szemeréd-Gowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A

More information

Properties of Indoor Received Signal Strength for WLAN Location Fingerprinting

Properties of Indoor Received Signal Strength for WLAN Location Fingerprinting Propertes of Indoor Receved Sgnal Strength for WLAN Locaton Fngerprntng Kamol Kaemarungs and Prashant Krshnamurthy Telecommuncatons Program, School of Informaton Scences, Unversty of Pttsburgh E-mal: kakst2,prashk@ptt.edu

More information

An Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems

An Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems STAN-CS-73-355 I SU-SE-73-013 An Analyss of Central Processor Schedulng n Multprogrammed Computer Systems (Dgest Edton) by Thomas G. Prce October 1972 Techncal Report No. 57 Reproducton n whole or n part

More information

+ + + - - This circuit than can be reduced to a planar circuit

+ + + - - This circuit than can be reduced to a planar circuit MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to

More information

DECOMPOSING ALLOCATIVE EFFICIENCY FOR MULTI-PRODUCT PRODUCTION SYSTEMS

DECOMPOSING ALLOCATIVE EFFICIENCY FOR MULTI-PRODUCT PRODUCTION SYSTEMS DECOMPOSING ALLOCATIVE EFFICIENCY FOR MULTI-PRODUCT PRODUCTION SYSTEMS EKONOMIKA A MANAGEMENT Tao Zhang Introducton Data envelopment analyss (DEA, the non-parametrc approach to measurng effcency, was frst

More information

Risk Model of Long-Term Production Scheduling in Open Pit Gold Mining

Risk Model of Long-Term Production Scheduling in Open Pit Gold Mining Rsk Model of Long-Term Producton Schedulng n Open Pt Gold Mnng R Halatchev 1 and P Lever 2 ABSTRACT Open pt gold mnng s an mportant sector of the Australan mnng ndustry. It uses large amounts of nvestments,

More information

Conversion between the vector and raster data structures using Fuzzy Geographical Entities

Conversion between the vector and raster data structures using Fuzzy Geographical Entities Converson between the vector and raster data structures usng Fuzzy Geographcal Enttes Cdála Fonte Department of Mathematcs Faculty of Scences and Technology Unversty of Combra, Apartado 38, 3 454 Combra,

More information

7.5. Present Value of an Annuity. Investigate

7.5. Present Value of an Annuity. Investigate 7.5 Present Value of an Annuty Owen and Anna are approachng retrement and are puttng ther fnances n order. They have worked hard and nvested ther earnngs so that they now have a large amount of money on

More information

An MILP model for planning of batch plants operating in a campaign-mode

An MILP model for planning of batch plants operating in a campaign-mode An MILP model for plannng of batch plants operatng n a campagn-mode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafe-concet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño

More information

Stochastic Protocol Modeling for Anomaly Based Network Intrusion Detection

Stochastic Protocol Modeling for Anomaly Based Network Intrusion Detection Stochastc Protocol Modelng for Anomaly Based Network Intruson Detecton Juan M. Estevez-Tapador, Pedro Garca-Teodoro, and Jesus E. Daz-Verdejo Department of Electroncs and Computer Technology Unversty of

More information

PRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIGIOUS AFFILIATION AND PARTICIPATION

PRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIGIOUS AFFILIATION AND PARTICIPATION PRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIIOUS AFFILIATION AND PARTICIPATION Danny Cohen-Zada Department of Economcs, Ben-uron Unversty, Beer-Sheva 84105, Israel Wllam Sander Department of Economcs, DePaul

More information

Portfolio Loss Distribution

Portfolio Loss Distribution Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets hold-to-maturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment

More information