WITH the increase in scale and complexity of process


 Clementine Marshall
 1 years ago
 Views:
Transcription
1 2052 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 Drect Causalty Detecton va the Transfer Entropy Approach Png Duan, Fan Yang, Member, IEEE, Tongwen Chen, Fellow, IEEE, and Srsh L. Shah, Member, IEEE Abstract The detecton of drect causalty, as opposed to ndrect causalty, s an mportant and challengng problem n root cause and hazard propagaton analyss. Several methods provde effectve solutons to ths problem when lnear relatonshps between varables are nvolved. For nonlnear relatonshps, currently only overall causalty analyss can be conducted, but drect causalty cannot be dentfed for such processes. In ths paper, we descrbe a drect causalty detecton approach sutable for both lnear and nonlnear connectons. Based on an extenson of the transfer entropy approach, a drect transfer entropy (DTE concept s proposed to detect whether there s a drect nformaton flow pathway from one varable to another. Especally, a dfferental drect transfer entropy concept s defned for contnuous random varables, and a normalzaton method for the dfferental drect transfer entropy s presented to determne the connectvty strength of drect causalty. The effectveness of the proposed method s llustrated by several examples, ncludng one expermental case study and one ndustral case study. Index Terms Dfferental transfer entropy, drect causalty, drect transfer entropy (DTE, nformaton flow pathway, kernel estmaton, normalzaton. NTE x ỹ Normalzed dscrete transfer entropy (NTE dsc from x to ỹ. NTE c x y Normalzed dfferental transfer entropy (NTE dff from x to y. NDTE c x y Normalzed dfferental drect transfer Entropy (NDTE dff from x to y. Eq. (1: Defnton of the TE dff from x to y. Eq. (2: Defnton of the TE dff from x to z. Eq. (3: Defnton of the TE dff from z to y. Eq. (4: Defnton of the DTE dff from x to y. Eq. (5: Defnton of the DTE dff from z to y. Eq. (6: Defnton of the TE dsc from x to ỹ. Eq. (10: Relatonshp between the TE dff from x to y and the TE dsc from x to ỹ. Eq. (11: Defnton of the DTE dsc from x to ỹ. Eq. (18: Lnear normalzaton of the TE dff. Eq. (19: Nonlnear normalzaton of the TE dff. Eq. (20: Normalzaton of the DTE dff. Eq. (21: Defnton of the DTE dff from x to y wth multple ntermedate varables. NOMENCLATURE x, y, z Contnuous random varables. x, ỹ, z Quantzed x, quantzed y, and quantzed z wth quantzaton bn szes x, y, and z, respectvely. T x y Dfferental transfer entropy(te dff from x to y. D x y Dfferental drect transfer entropy (DTE dff from x to y. t x ỹ Dscrete transfer entropy (TE dsc from x to ỹ. d x ỹ Dscrete drect transfer entropy (DTE dsc from x to ỹ. Manuscrpt receved May 2, 2012; revsed October 11, 2012; accepted December 2, Manuscrpt receved n fnal form December 7, Date of publcaton January 9, 2013; date of current verson October 15, Ths work was supported by an NSERC Strategc Project, an NSFC Project under Grant and the Tsnghua Natonal Laboratory for Informaton Scence and Technology CrossDscplne Foundaton. Recommended by Assocate Edtor J. Yu. P. Duan and T. Chen are wth the Department of Electrcal and Computer Engneerng, Unversty of Alberta, Edmonton, AB T6G 2V4, Canada (emal: F. Yang s wth the Department of Automaton, Tsnghua Natonal Laboratory for Informaton Scence and Technology, Tsnghua Unversty, Bejng , Chna (emal: S. L. Shah s wth the Department of Chemcal and Materals Engneerng, Unversty of Alberta, Edmonton, AB T6G 2G6, Canada (emal: Dgtal Object Identfer /TCST IEEE I. INTRODUCTION WITH the ncrease n scale and complexty of process operatons, the detecton and dagnoss of plantwde abnormaltes and dsturbances are major problems n the process ndustry. Compared wth the tradtonal fault detecton, fault detecton and dagnoss n a largescale complex system are partcularly challengng because of the hgh degree of nterconnectons among dfferent parts n the system. A smple fault may easly propagate along nformaton and materal flow pathways and affect other parts of the system. To determne the root cause(s of certan abnormalty, t s mportant to capture the process connectvty and fnd the connectng pathways. A qualtatve process model n the form of a dgraph has been wdely used n root cause and hazard propagaton analyss [1]. Dgraphbased models usually express the causal relatonshps between faults and symptoms and defne the fault propagaton pathways by ncorporatng expert knowledge of the process [2]. A drawback s that extractng expert knowledge s very tme consumng and that knowledge s not always easly avalable [3]. The modelng of dgraphs can also be based on mathematcal equatons [4], [5], yet for largescale complex processes t s dffcult to establsh practcal and precse mathematcal models. Datadrven methods provde another way to fnd the causal relatonshps between process varables. A few databased
2 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2053 methods are capable of detectng the causal relatonshps for lnear processes [6]. In the frequency doman, drected transfer functons (DTFs [7] and partal drected coherence (PDC [8] are wdely used n bran connectvty analyss. Other methods such as Granger causalty [9], path analyss [10], and crosscorrelaton analyss wth lagadjusted varables [11] are commonly used. The predctablty mprovement based on the nearest neghbors s proposed as an asymmetrcal measure of nterdependence n bvarate tme seres and appled to quantfy the drectonal nfluences among physologcal sgnals [12] and also ndustral processes varables [13], [14]. Informaton theory provdes a wde varety of approaches for measurng causal nfluence among multvarate tme seres [15]. Based on transton probabltes contanng all nformaton on causalty between two varables, the transfer entropy (TE was proposed to dstngush between drvng and respondng elements [16] and s sutable for both lnear and nonlnear relatonshps; t has been successfully used n chemcal processes [17] and neuroscences [18]. TE has two forms, dscrete TE (TE dsc for dscrete random varables [16], and dfferental TE (TE dff for contnuous random varables [19]. It has been shown n [20] that, for Gaussan dstrbuted varables wth lnear relatonshps, Granger causalty and TE are equvalent. The equvalence of the two causalty measures has been extended under certan condtons on probablty densty dstrbutons of the data [21]. In [22], comparsons are gven for several causalty detecton methods; these methods nclude TE, extended and nonlnear Granger causalty, and predctablty mprovement. That paper also ncludes a dscusson on the usefulness of the methods for detectng asymmetrc couplngs and nformaton flow drectons n the determnstc chaotc systems. The authors conclude that, gven a complex system wth aprorunknown dynamcs, the frst method of choce mght be TE. If a large number of samples are avalable, the alternatve methods mght be nonlnear Granger causalty and predctablty mprovement. Snce nformaton flow specfcally means how varaton propagates from one varable to another, t s valuable to detect whether the causal nfluence between a par of varables s along a drect pathway wthout any ntermedate varables or ndrect pathways through some ntermedate varables. In the frequency doman, a DTF/PDCbased method for quantfcaton of drect and ndrect energy flow n a multvarate process was recently proposed [23]. Ths method was based on vector autoregressve or vector movngaverage model representatons, whch are sutable for lnear multvarate processes. In the tme doman, a path analyss method was used to calculate the drect effect coeffcents [24]. The calculaton was based on a regresson model of the varables, whch captures only lnear relatonshps. For both lnear and nonlnear relatonshps, based on a multvarate verson of TE, the partal TE was proposed to quantfy the total amount of ndrect couplng medated by the envronment and was successfully used n neuroscences [25]. In [25]. The partal TE s defned such that all the envronmental varables are consdered as ntermedate varables, whch s not necessary n most cases; and n any case, ths wll ncrease the computatonal burden sgnfcantly. On the other hand, the utlty of the partal TE s to detect undrectonal causaltes, whch s sutable for neuroscences; however, n ndustral processes, feedback and bdrectonal causaltes, due to recycle streams, are common. Thus, the partal TE method cannot be drectly used for drect/ndrect causalty detecton n the process ndustry. The man contrbuton of ths paper s a TEbased methodology to detect and dscrmnate between drect and ndrect causalty relatonshps between process varables of both lnear and nonlnear multvarate systems. Specfcally, ths method s able to uncover explct drect and ndrect, as f through ntermedate varables, connectvty pathways between varables. The rest of ths paper s organzed as follows. In Secton II, we apply the TE dff for contnuous random varables to detect total causalty and defne a dfferental drect transfer entropy (DTE dff 1 to detect drect causalty. Calculaton methods and the normalzaton methods are proposed for both the TE dff and the DTE dff n the same secton. Secton III descrbes three examples to show the effectveness of the proposed drect causalty detecton method. An expermental case study and an ndustral case study are ntroduced n Secton IV to show the usefulness of the proposed method for detectng drect/ndrect connectng pathways, followed by concludng remarks n Secton V. II. DETECTION OF DIRECT CAUSALITY In ths secton, an extenson of the TE drect transfer entropy (DTE s proposed to detect the drect causalty between two varables. In addton to ths, calculaton methods and the normalzaton methods are also presented for the TE and the DTE, respectvely. A. DTE In order to determne the nformaton and materal flow pathways to construct a precse topology of a process, t s mportant to determne whether the nfluence between a par of process varables s along drect or ndrect pathways. The drect pathway means drect nfluence wthout any ntermedate or confoundng varables. The TE measures the amount of nformaton transferred from one varable x to another varable y. Ths extracted transfer nformaton represents the total causal nfluence from x to y. It s dffcult to dstngush whether ths nfluence s along a drect pathway or ndrect pathways through some ntermedate varables. For example, gven three varables x, y, andz, f the calculated transfer entropes from x to y, from x to z, and from z to y are all larger than zero, then we can conclude that x causes y, x causes z, andz causes y. We can also conclude that there s an ndrect pathway from x to y va the ntermedate varable z whch transfers nformaton from x to y. However, we cannot dstngush whether there s a drect pathway from x to y (see Fg. 1, because t s possble that there exst both a drect pathway from x to y and an ndrect pathway va the ntermedate varable z. 1 We cauton the reader to be aware of the term DTE dff for dfferental drect transfer entropy and that t s dfferent from the term dscrete drect transfer entropy (DTE dsc as t apples to dscrete random varables.
3 2054 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 x Fg. 1. Detecton of drect causalty from x to y. z? In order to detect the drect and ndrect pathways of the nformaton transfer, the defnton of a DTE s ntroduced as follows. Snce the data analyzed here s unformly sampled data, as obtaned from processes that are contnuous, we only consder contnuous random varables n ths paper. Gven three contnuous random varables x, y, andz, let them be sampled at tme nstants and denoted by x [x mn, x max ], y [y mn, y max ],andz [z mn, z max ] wth = 1, 2,...,N, where N s the number of samples. The causal relatonshps between each par of these varables can be estmated by calculatng the TEs [16]. Let y +h1 denote the value of y at tme nstant + h 1,that s, h 1 steps n the future from, andh 1 s referred to as the predcton horzon; y (k 1 =[y, y τ1,...,y (k1 1τ 1 ] and x (l 1 =[x, x τ1,...,x (l1 1τ 1 ] denote embeddng vectors wth elements from the past values of y and x, respectvely (k 1 s the embeddng dmenson of y and l 1 s the embeddng dmenson of x; τ 1 s the tme nterval that allows the scalng n tme of the embedded vector, whch can be set to be h 1 = τ 1 4 as a rule of thumb [17]; f (y +h1, y (k 1, x (l 1 denotes the jont probablty densty functon (pdf, and f ( denotes the condtonal pdf, and thus f (y +h1 y (k 1, x (l 1 denotes the condtonal pdf of y +h1 gven y (k 1 and x (l 1 and f (y +h1 y (k 1 denotes the condtonal pdf of y +h1 gven. The dfferental TE (TE dff from x to y, for contnuous varables, s then calculated as follows: T x y = f (y +h1, y (k 1, x (l 1 y (k 1 log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k dw (1 1 where the base of the logarthm s 2 and w denotes the random vector [y +h1, y (k 1, x (l 1 ]. By assumng that the elements of w are w 1,w 2,...,w s, ( dw denotes ( dw 1 dw s for smplcty, and the followng notatons have the same meanng as ths one. The TE from x to y can be understood as the mprovement when usng the past nformaton of both x and y to predct the future of y compared to only usng the past nformaton of y. In other words, the TE represents the nformaton about a future observaton of varable y obtaned from the smultaneous observatons of past values of both x and y, after dscardng the nformaton about the future of y obtaned from the past values of y alone. Smlarly, the TE dff from x to z s calculated as follows: T x z = f (z +h2, z (m 1, x (l 2 log f (z +h 2 z (m 1, x (l 2 f (z +h2 z (m 1 y dη (2 where h 2 s the predcton horzon, z (m 1 = [z, z τ2,..., z (m1 1τ 2 ] and x (l 2 =[x, x τ2,...,x (l2 1τ 2 ] are embeddng vectors wth tme nterval τ 2,andη denotes the random vector [z +h2, z (m 1, x (l 2 ]. The TE dff from z to y s calculated as follows: T z y = f (y +h3, y (k 2, z (m 2 log f (y +h 3 y (k 2, z (m 2 f (y +h3 y (k dζ (3 2 where h 3 s the predcton horzon, y (k 2 = [y, y τ3,..., y (k2 1τ 3 ] and z (m 2 = [z, z τ3,...,z (m2 1τ 3 ] are embeddng vectors wth tme nterval τ 3,andζ denotes the random vector [y +h3, y (k 2, z (m 2 ]. If T x y, T x z,andt z y are all larger than zero, then we conclude that x causes y, x causes z, andz causes y. Inths case, we need to dstngush whether the causal nfluence from x to y s only va the ndrect pathway through the ntermedate varable z, or, n addton to ths, there s another drect pathway from x to y, as shown n Fg. 1. We defne a drect causalty from x to y as x drectly causng y, whch means there s a drect nformaton and/or materal flow pathway from x to y wthout any ntermedate varables. In order to detect whether there s a drect causalty from x to y, we defne a dfferental DTE (DTE dff from x to y as follows: D x y = f (y +h, y (k, z (m 2 +h h 3, x (l 1 +h h 1 log f (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 f (y +h y (k, z (m dv (4 2 +h h 3 where v denotes the random vector [y +h, y (k, z (m 2 +h h 3, x (l 1 +h h 1 ]; the predcton horzon h s set to be h = max(h 1, h 3 ;fh = h 1,theny (k = y (k 1,fh = h 3,theny (k = y (k 2 ; the embeddng vector z (m 2 +h h 3 =[z +h h3, z +h h3 τ 3,...,z +h h3 (m 2 1τ 3 ] denotes the past values of z whch can provde useful nformaton for predctng the future y at tme nstant + h, where the embeddng dmenson m 2 and the tme nterval τ 3 are determned by (3; the embeddng vector x (l 1 +h h 1 = [x +h h1, x +h h1 τ 1,...,x +h h1 (l 1 1τ 1 ] denotes the past values of x whch can provde useful nformaton to predct the future y at tme nstant +h, where the embeddng dmenson l 1 and the tme nterval τ 1 are determned by (1. Note that the parameters n DTE dff are all determned by the calculaton of the TEs for consstency. The DTE dff represents the nformaton about a future observaton of y obtaned from the smultaneous observaton of past values of both x and z, after dscardng the nformaton about the future y obtaned from the past z alone. Ths can mean that f the pathway from z to y s cut off, wll the hstory of x stll provde some helpful nformaton to predct the future y? Obvously, f ths nformaton s nonzero (greater than zero, then there s a drect pathway from x to y. Otherwse,theres no drect pathway from x to y, and the causal nfluence from x to y s all along the ndrect pathway va the ntermedate varable z.
4 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2055 Note that the drect causalty here s a relatve concept; snce the measured process varables are lmted, the drect causalty analyss s only based on these varables. In other words, even f there are ntermedate varables n the connectng pathway between two measured varables, as long as none of these ntermedate varables s measured, we stll state that the causalty s drect between the par of measured varables. After the calculaton of D x y, f there s drect causalty from x to y, we need to further judge whether the causalty from z to y s true or spurous, because t s possble that z s not a cause of y and the spurous causalty from z to y s generated by x,.e., x s the common source of both z and y. As shown n Fg. 2, there are stll two cases of the nformaton flow pathways between x, y, andz, andthe dfference s whether there s true and drect causalty from z to y. Thus, DTE dff from z to y needs to be calculated D z y = p(y +h, y (k, x (l 1 +h h 1, z (m 2 +h h 3 log p(y +h y (k, x (l 1 +h h 1, z (m 2 +h h 3 p(y +h y (k, x (l dv (5 1 +h h 1 where the parameters are the same as n (4. If d z y > 0, then there s true and drect causalty from z to y, asshown n Fg. 2(a. Otherwse, the causalty from z to y s spurous, whch s generated by the common source x, asshownn Fg. 2(b. The need for detecton of drect and ndrect causalty based on measured process varables s dscussed. The tradtonal TE method only determnes whether there s causalty from x to y, but we cannot tell whether the causal nfluence s along a drect pathway or ndrect pathways through some ntermedate varables (see Fg. 1. The purpose of process causalty analyss s to nvestgate propagaton of faults, alarms events, and sgnals through materal and nformaton flow pathways (for example va feedback control and n ths respect t s mportant to know whether the connecton between varables of nterest s drect or ndrect. As shown n Fg. 1, f drect causalty from x to y s detected, then there should be a drect nformaton and/or materal flow pathway from x to y. Otherwse, there s no drect nformaton and/or materal flow pathway from x to y and the drect lnk should be elmnated. Ths s clearly llustrated n the expermental threetank case study presented n Secton IV. Such cases are common n ndustral processes; the tradtonal TE approach wll reveal a myrad of connectons, as t s not able to dscrmnate between drect and ndrect causalty, whereas once one s able to detect drect paths, the number of connectng pathways reduces sgnfcantly. Another mportant case s to detect the true or spurous causalty as shown n Fg. 2. In fact, ths can tell whether there s a drect nformaton and/or materal flow pathway from z to y or there s no nformaton flow pathway from z to y at all. If we only use the tradtonal TE method, we may conclude that there s causal nfluence from z to y and therefore there s an nformaton flow pathway from z to y, whch s not true because they are both nfluenced by a common cause. Thus, (a Fg. 2. Informaton flow pathways between x, y, andz wth (a true and drect causalty from z to y, and (b spurous causalty from z to y (meanng that z and y have a common perturbng source, x, and therefore they may appear to be connected or correlated even when they are not connected physcally. the detecton of drect and ndrect causalty s necessary for capturng the true process connectvty. An mportant applcaton of causalty analyss for capturng process connectvty s to fnd the fault propagaton pathways and dagnose the root cause of certan dsturbance or faults. If we only detect causalty va the tradtonal TE approach, total causalty and spurous causalty would be detected to yeld an overly complcated set of pathways from whch root cause dagnoss of faults would be dffcult f not erroneous. However, f we are able to dfferentate between drect and ndrect, true and spurous causalty, then the derved causal map may be much smpler and more accurate to tell the fault propagaton pathways and whch varable s the lkely root cause. Ths pont s clearly llustrated by the ndustral case study presented n Secton IV. B. Relatonshps Between DTE dff and DTE dsc The TE dff and the DTE dff mentoned above are defned for contnuous random varables. For contnuous random varables, a wdely used TE calculaton procedure s to perform quantzaton frst and then use the formula of TE dsc [17]. Thus, we need to establsh a connecton between ths quantzatonbased procedure and the TE dff procedure. For the contnuous random varables x, y,andz,let x, ỹ and z denote the quantzed x, y, andz, respectvely. Assume that the supports of x, y, andz,.e.,[x mn, x max ], [y mn, y max ],and [z mn, z max ], are classfed nto n x, n y,andn z nonoverlappng ntervals (bns, respectvely, and the correspondng quantzaton bn szes of x, y, andz are x, y,and z, respectvely. Takng x for an example, f we choose a unform quantzer, then we have x = x max x mn. n x 1 We can see that the quantzaton bn sze s related to the varable support and the number of quantzaton ntervals (bn number. Gven a varable support, the larger the bn number, the smaller s the quantzaton bn sze. After quantzaton, the TE from x to y can be approxmated by the TE dsc from x to ỹ t x ỹ = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h 1 ỹ (k 1, x (l 1 p(ỹ +h1 ỹ (k (6 1 where the sum symbol represents k 1 + l 1 + 1sumsoverall ampltude bns of the jont probablty dstrbuton and condtonal probabltes; ỹ (k 1 =[ỹ, ỹ τ1,...,ỹ (k1 1τ 1 ] and =[ x, x τ1,..., x (l1 1τ 1 ] denote embeddng vectors; x (l 1 (b
5 2056 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 p(ỹ +h1, ỹ (k 1, x (l 1 denotes the jont probablty dstrbuton; and p( denotes the condtonal probabltes. The meanng of other parameters remans unchanged. From (6 we can express TE dsc usng condtonal Shannon entropes [26] by expandng the logarthm, as t x ỹ = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h 1, ỹ (k 1, x (l 1 p(ỹ (k 1, x (l 1 p(ỹ +h1, ỹ (k 1 log p(ỹ +h 1, ỹ (k 1 p(ỹ (k 1 = H (ỹ +h1 ỹ (k 1 H (ỹ +h1 ỹ (k 1, x (l 1 (7 where H (ỹ +h1 ỹ (k 1 = p(ỹ +h1, ỹ (k 1 log p(ỹ +h1 ỹ (k 1 and H (ỹ +h1 ỹ (k 1, x (l 1 = p(ỹ +h1, ỹ (k 1, x (l 1 log p(ỹ +h1 ỹ (k 1, x (l 1 are the condtonal Shannon entropes. Smlar to TE dsc, we can express the TE dff usng dfferental condtonal entropes, as T x y = f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw f (y +h1, y (k 1 log f (y +h1 y (k 1 du = H c (y +h1 y (k 1 H c (y +h1 y (k 1, x (l 1 (8 where u denotes the random vector [y +h1, y (k 1 ], and H c (y +h1 y (k 1 and H c (y +h1 y (k 1, x (l 1 are the dfferental condtonal entropes. Theoretcally, as the bn szes approach zero, the probablty p(ỹ +h1, ỹ (k 1, x (l 1 n (7 can be approxmated by y k 1 y l 1 x f (y +h1, y (k 1, x (l 1. Thenwehave lm x, y 0 t x ỹ = lm x, y 0 { y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 log y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 k 1 y l 1 x f (y (k 1, x (l 1 y k 1 y f (y +h1, y (k 1 log y k 1 y f (y +h1, y (k 1 } k 1 y f (y (k 1 { = lm y k 1 y x, y 0 l 1 x f (y +h1, y (k 1, x (l 1 ( log y + log f (y +h1 y (k 1, x (l 1 y k 1 y f (y +h1, y (k 1 ( log y + log f (y +h1 y (k 1 }. (9 As x, y 0, we have y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 f (y +h1, y (k 1, x (l 1 dw = 1, y k 1 y f (y +h1, y (k 1 f (y +h1, y (k 1 du = 1 and the ntegral of the functon f ( log f ( can be approxmated n the Remannan sense by y k 1 y l 1 x f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw y k 1 y f (y +h1, y (k 1 log f (y +h1 y (k 1 f (y +h1, y (k 1 log f (y +h1 y (k 1 du. Thus lm t x ỹ x, y 0 = lm log y y 0 + f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw lm log y y 0 f (y +h1, y (k 1 log f (y +h1 y (k 1 du = f (y +h1, y (k 1, x (l 1 log f (y +h1 y (k 1, x (l 1 dw f (y +h1, y (k 1 log f (y +h1 y (k 1 du = f (y +h1, y (k 1, x (l 1 log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k dw 1 = T x y. (10 Ths means that the dfferental TE from x to y s the same as the dscrete TE from quantzed x to quantzed y n the lmt as the quantzaton bn szes of both x and y approach zero. Remark: From (9 and (10, we can see that the dfference between the dfferental condtonal entropy and the lmtng value of the Shannon condtonal entropy as x, y 0 s an nfnte offset, lm y 0 log y. Thus, the dfferental condtonal entropy can be negatve. Smlar to TE, the DTE from x to y can be approxmated by a dscrete DTE (DTE dsc from x to ỹ d x ỹ = p(ỹ +h, ỹ (k, z (m 2 +h h 3, x (l 1 +h h 1 log p(ỹ +h ỹ (k, z (m 2 +h h 3, x (l 1 +h h 1 p(ỹ +h ỹ (k, z (m 2 +h h 3 (11 where ỹ (k, z (m 2 +h h 3,and x (l 1 +h h 1 are embeddng vectors of ỹ, z, and x, respectvely. The defntons of the other quanttes are smlar to that n (4.
6 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2057 X j, j = 1, 2,...,m, and μ = m j=1 μ j /m; then the standard error of the estmated mean μ s gven by mj=1 (μ j μ 2 σ = m(m 1 Fg. 3. Relatonshps between TEs and DTEs. RV means random varable. For the DTE dff and the DTE dsc, usng the same proof procedure wth the TE, we can obtan lm x, y, z 0 d x ỹ = D x y whch means that the DTE dff from x to y s the same as the DTE dsc from quantzed x to quantzed y n the lmt as the quantzaton bn szes of x, y, and the ntermedate varable z approach zero. Fg. 3 llustrates the relatonshps between TE dff and TE dsc, and between DTE dff and DTE dsc. It should be noted that the smaller the bn sze, the more accurate s the quantzaton and the closer are DTE dsc and DTE dff. Note that the computatonal burden of the summaton and the probablty estmaton n (6 and (11 wll ncrease sgnfcantly wth ncreasng quantzaton bn numbers,.e., n x, n y,andn z. Thus, for the choce of bn szes, there s a tradeoff between the quantzaton accuracy and the computatonal burden n TE dsc and DTE dsc calculatons. In practce, the condtons that the quantzaton bn szes approach zero are dffcult to satsfy. Thus, n order to avod the roundoff error of quantzaton, we drectly use TE dff and DTE dff to calculate TE and DTE, respectvely. C. Calculaton Method 1 Requred Assumptons for the DTE Calculaton: Snce the concept of DTE s an extenson of TE, the requred assumptons for DTE s exactly the same as TE; the collected sampled data must be statonary n a wde sense wth a large data length, preferably no less than 2000 observatons [17]. Statonarty requres that the dynamcal propertes of the system must not change durng the observaton perod. Snce n most cases we do not have drect access to the system and we cannot establsh evdence that ts parameters are ndeed constant, we have to test for statonarty based on the avalable dataset. For the purpose of testng for statonarty, the smplest and most wdely used method s to measure the mean and the varance for several segments of the dataset (equvalent to an ergodcty test and then use a standard statstcal hypothess test to check whether the mean and the varance change. More subtle quanttes such as spectral components, correlatons, or nonlnear statstcs may be needed to detect less obvous nonstatonarty [27]. In ths paper, we use the mean and varance to test for statonarty. We dvde a gven dataset, denoted by x, = 1, 2,...,N, nto m consecutve segments, denoted by X 1, X 2,...,X m, each contanng s data ponts. Let μ j denote the mean value of where the standard devaton dvded by an extra m s the error when estmatng the mean value of Gaussan dstrbuted uncorrelated numbers [27]. The null hypothess for statonarty testng s that the dataset s statonary. The sgnfcance level for the mean testng s defned as μ j μ > 6 for j = 1, 2,...,m. (12 σ A sxsgma threshold for the sgnfcance level s chosen here. Specfcally, f there exsts μ j > μ + 6σ or μ j < μ 6σ for j = 1, 2,...,m, then the null hypothess that the dataset s statonary s rejected. If μ 6σ < μ j < μ + 6σ holds for all js, then the null hypothess s accepted that the dataset s statonary. For the varance test, let ˆx, = 1, 2,...,N denote the normalzed dataset of x, and x 1, x 2,..., x m denote the correspondng consecutve segments. Then we have x j = ˆx s( j 1+1, ˆx s( j 1+2,..., ˆx sj for j = 1, 2,...,m. Snce the sum of squares of the elements n each segment has the chsquared dstrbuton wth s degrees of freedom ˆv j = ˆx s( 2 j 1+1 +ˆx s( 2 j ˆx sj 2 χ s 2, we can check whether or not the dataset s statonary by comparng ˆv j wth χs 2(α. If there exsts ˆv j > χs 2 (α for j = 1, 2,...,m, then the null hypothess that the dataset s statonary s rejected wth (1 α 100% confdence. If ˆv j <χs 2 (α for all js, then the null hypothess s accepted. Multmodalty s often encountered n ndustral processes due to the normal operatonal changes as well as changes n the producton strategy [28]. For such multmodal processes, a dataset wth a large number of samples s most lkely to be nonstatonary as the data would reflect transtons from one mode to another, whereas a key assumpton of the TE/DTE method s statonarty of the sampled data. In order to handle the process multmodalty, one would have to partton the data nto dfferent segments correspondng to dfferent modes. A few tmeseres analyss methods [29], [30] have been proposed for segmentaton of tme seres to determne when the process mode has changed. As long as the segments correspondng to dfferent modes are obtaned, we can detect (drect causalty for each mode of the process usng the approprate segment. Note that the causal relatonshps may change wth mode swtchng of the process. 2 Estmaton of the TE dff and the DTE dff : For the TE from x to y, snce (1 can be wrtten as { T x y = E t can be approxmated by log f (y +h 1 y (k 1, x (l 1 f (y +h1 y (k 1 N h 1 1 T x y = log f (y +h 1 y (k 1, x (l 1 N h 1 r + 1 =r f (y +h1 y (k 1 } (13
7 2058 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 where N s the number of samples and r = max{(k 1 1 τ 1 + 1,(l 1 1τ 1 + 1}. Just as wth TE dff,thedte dff (4 can be wrtten as {log f (y +h y (k, z (m 2 +h h 3, x (l } 1 +h h 1 D x y = E whch can be approxmated by D x y = 1 N h j + 1 N h = j f (y +h y (k, z (m 2 +h h 3 log f (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 f (y +h y (k, z (m 2 +h h 3 (14 where j = max{(k 1 1τ 1 + 1,(k 2 1τ 3 + 1, h + h 3 + (m 2 1τ 3 + 1, h + h 1 + (l 1 1τ 1 + 1}. 3 Kernel Estmaton of pdfs: In (13 and (14, the condtonal pdfs are expressed by the jont pdfs and then obtaned by the kernel estmaton method [31]. Here, the followng Gaussan kernel functon s used k(u = 1 2π e 1 2 u2. Then a unvarate pdf can be estmated by f ˆ(x = 1 N ( x X k Nγ γ =1 (15 where N s the number of samples, and γ s the bandwdth chosen to mnmze the mean ntegrated squared error of the pdf estmaton and calculated by γ = 1.06σ N 1/5 accordng to the normal reference ruleofthumb [31], [32], where σ s the standard devaton of the sampled data {X } =1 N. For qdmensonal multvarate data, we use the Fukunaga method [31] to estmate the jont pdf. Suppose that X 1,...,X N consttute a qdmensonal vector (X R q wth a common pdf f (x 1, x 2,...,x q. Let x denote the qdmensonal vector [x 1, x 2,...,x q ] T ; then the kernel estmaton of the jont pdf s f ˆ (det S 1/2 (x = NƔ q N K =1 { } Ɣ 2 (x X T S 1 (x X (16 where Ɣ s smlar to the bandwdth γ n (15. The estmated jontpdfs smootherwhenɣ s larger. However, a substantally larger Ɣ s most lkely to result n an naccurate estmaton. Thus, Ɣ s also chosen to mnmze the mean ntegrated squared error of the jont pdf estmaton and calculated by Ɣ = 1.06N 1/(4+q. S s the covarance matrx of the sampled data, and K s the Gaussan kernel satsfyng K (u = (2π q/2 e 1 2 u. Note that when q = 1, (16 s smplfed nto (15. For the TE, the estmaton of the computatonal complexty s dvded nto two parts: the kernel estmaton of the pdf usng (16, and the calculaton of the TE dff usng (13. For each jont pdf of dmenson q, the computatonal complexty s O(N 2 q 2. Consderng the condtonal pdfs are estmated by the jont pdfs, the maxmum dmenson of the jont pdf s k 1 + l and, thus, the computatonal complexty for the pdf estmaton s O(N 2 (k 1 + l 1 2. For calculaton of the TE dff n (13, approxmately N summatons are requred. Thus, the total computatonal complexty for the TE dff s O(N 2 (k 1 + l 1 2. Smlarly, we can obtan that the computatonal complexty for the DTE dff usng (14 s O(N 2 (k + m 2 + l 1 2. It s obvous that the number of samples and the embeddng dmensons determne the computng speed. Snce the samples number s preferred to be no less than 2000 observatons [17], we need to lmt the choce of the embeddng dmensons. The detals on how to choose the embeddng dmensons are gven n the followng subsecton. Note that the computatonal complexty s relatvely large because of the kernel estmaton of the (jont pdfs, and that the computatonal complexty for the DTE ncreases wth an ncreasng number of ntermedate varables. Therefore, one would have to apply the method to smaller unts wth a smaller number of varables. A largescale complex system can be broken down nto smaller unts and thereafter analyzed for causal relatonshps wthn each unt and between dfferent unts, and fnally the nformaton flow pathways of the whole process can be establshed. 4 Determnaton of the Parameters of the TE: In the use of the TE approach to detect causalty, there are four undetermned parameters: 1 the predcton horzon (h 1 ; 2 the tme nterval (τ 1 ; 3 and the embeddng dmensons (k 1 and l 1. Snce these four parameters greatly affect the calculaton results of the transfer entropes, we need to fnd a systematc method to determne them. Frst, snce h 1 = τ 1 4 as a rule of thumb [17], we can further set ntal values for h 1 and τ 1 accordng to apror knowledge of the process. For example, we start by settng the ntal values for h 1 = τ 1 = 1. Second, we can determne the embeddng dmenson of y,.e., the wndow sze of the hstorcal y used for the future y predcton. The embeddng dmenson of y,.e., k 1, can be determned as the mnmum nonnegatve nteger above whch the change rate of H c (y +h1 y (k 1 decreases sgnfcantly. Consderng a large k 1 can ncrease the dmenson of the jont pdf and the dffculty n pdf estmaton, f k 1 s greater than 3, we need to ncrease h 1 and τ 1 and repeat the calculaton untl a k 1 3 s found to make the change rate of H c (y +h1 y (k 1 decrease sgnfcantly. Fnally, we can determne the embeddng dmenson of x,.e., the wndow sze of the hstorcal x used for the future y predcton. Based on the values of k 1, h 1,, and τ 1, the embeddng dmenson of x,.e., l 1, s determned as the mnmum postve nteger above whch the change rate of the TE from x to y decreases sgnfcantly. 5 Normalzaton: It s easy to prove that both the TE and the DTE are condtonal mutual nformaton; thus they are always nonnegatve. However, small values of the TE and the DTE suggest no causalty or drect causalty whle large values do. In order to quantfy the strength of the total causalty and drect causalty, normalzaton s necessary.
8 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE 2059 In [33], the normalzed dscrete TE (NTE dsc sdefnedas NTE x ỹ = t x ỹ t shuffled x ỹ H (ỹ +h1 ỹ (k [0, 1] (17 1 where t shuffled x ỹ s an estmate of the same TE n shuffled data of x and ỹ. ThsNTE dsc ntutvely represents the fracton of nformaton n ỹ not explaned by ts own past but explaned by the past of x. Eq. (17 s sutable for the normalzaton of the TE dsc. For TE dff, we cannot just substtute H (ỹ +h1 ỹ (k 1 wth the dfferental condtonal entropy H c (y +h1 y (k 1, snce H c (y +h1 y (k 1 could be negatve. Moreover, usng shuffled data to elmnate the calculaton bas s not accurate because random shufflng may destroy the statstcal propertes of the tme seres. Also, t shuffled x ỹ s an average of transfer entropes obtaned on n trals. To obtan a better result, n should be large enough, whch wll ncrease the computatonal burden sgnfcantly. Thus, we need to propose a new normalzaton method for TE dff. In (17 the zero pont s regarded as the orgn and t represents a determnstc varable. For dfferental entropy, the value nstead of zero means that the varable s determnstc. The maxmal dfferental entropy gven a fnte support s n the form of a unform dstrbuton [34]. So, we defne the orgn as the maxmal dfferental entropy of y wth the unform dstrbuton ymax 1 H 0 (y = log dy y max y mn y max y mn = log(y max y mn y mn 1 where y max and y mn denote the maxmum and mnmum values of the varable y, respectvely. Consderng that the TE dff s the dfference between two dfferental condtonal entropes, as shown n (8, we defne the normalzed dfferental TE (NTE dff as NTE c x y = H c (y +h1 y (k 1 H c (y +h1 y (k 1, x (l 1 H 0 H c (y +h1 y (k 1, x (l 1 T x y = H 0 H c (y +h1 y (k 1, x (l 1 [0, 1]. (18 Intutvely, the numerator term represents the TE to capture the nformaton about y not explaned by ts own hstory and yet explaned by the hstory of x; the denomnator term represents the nformaton n y that s provded by the past values of both x and y. It s obvous that NTE c x y = 0f T x y = 0. If y s unformly dstrbuted and the nformaton about y explaned by the hstory of both x and y s completely explaned by the hstory of x, whch means H c (y +h1 y (k 1 = H 0, then accordng to (18 we obtan NTE c x y = 1. Snce an entropy H represents the average number of bts needed to optmally encode ndependent draws of a random varable [16], the uncertan nformaton contaned n a sgnal s n fact proportonal to 2 H. Here, a sgnal means a specfc realzaton of the random varable. We extend the lnear normalzaton functon n (18 to a nonlnear functon as follows: NTE c x y = 2H c (k (y +h1 y 1 2 H c (y +h1 y (k 1,x (l 1 2 H 0 2 H c (y +h1 y (k 1,x (l 1 [0, 1]. (19 The meanng of (19 s the same as that n (18. Ths nonlnear normalzaton functon (19 wll be used later. Snce the DTE dff n (4 represents the nformaton drectly provded from the past x to the future y, a normalzed dfferental DTE (NDTE dff sdefnedas D x y NDTE c x y = H c (y +h y (k H c (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 [0, 1 (20 where H c (y +h y (k and H c (y +h y (k, z (m 2 +h h 3, x (l 1 +h h 1 are the dfferental condtonal entropes. Intutvely, ths NDTE dff represents the percentage of drect causalty from x to y n the total causalty from both x and z to y. D. Extenson to Multple Intermedate Varables The defnton of the DTE dff from x to y can be easly extended to multple ntermedate varables z 1, z 2,...,z q,as D x y = f (y +h, y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 +h h 1 log f (y +h y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 f (y +h y (k, z (s 1 1, 1,...,z (s q q, q +h h 1 dξ (21 where s 1,...,s q and 1,..., q are the correspondng parameters determned by the calculatons of the transfer entropes from z 1,...,z q to y, and ξ denotes the random vector [y +h, y (k, z (s 1 1, 1,...,z (s q q, q, x (l 1 +h h 1 ]. If d x y s zero, then there s no drect causalty from x to y, and the causal effects from x to y are all along the ndrect pathways va the ntermedate varables z 1, z 2,...,z q.ifd x y s larger than zero, then there s drect causalty from x to y. The formulatons of the proposed DTE and the partal TE n [25] are smlar, but the basc deas are qute dfferent. The major dfference s that for the partal TE, all the envronmental varables are consdered as ntermedate varables, whereas for the DTE, the ntermedate varables are chosen based on calculaton results from the tradtonal TE. Specfcally, the partal TE was proposed as a substtuton of the tradtonal TE. We can choose ether tradtonal TE to detect total causalty or partal TE to detect partal causalty only. However, the DTE was proposed here as an extenson of the tradtonal TE and should be used after capturng the nformaton flow pathways va the tradtonal TE method. The ntermedate varables from x to y are determned as the varables wthn the nformaton flow pathway from x to y (see Fg. 1 and common sources of both x and y (see Fg. 2. More specfc comparsons between the partal TE and the DTE are as follows:
9 2060 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER The partal TE s defned such that all the envronmental varables are consdered as ntermedate varables, whch s not necessary n most cases and n any case ths wll ncrease the computatonal burden sgnfcantly. Moreover, ths may even result n false causalty detecton. For example, gven three varables x, y, andz, assume that x and y are ndependent and the true causal relatonshp between them s that x causes z and y also causes z,.e., x z y. It s clear that, gven the nformaton of z, x and y are no longer ndependent. If we use the partal TE to detect the causalty from x to y, gven nformaton of z, t s most lkely to conclude that there s causalty between x and y as long as there s a tme delay between them, although n fact the detected causalty between x and y does not exst. Whle usng the DTE method, we need to frst detect causalty by usng the tradtonal TE method, and then determne whch varable s the ntermedate varable. Snce the nformaton of z s unknown whle usng the tradtonal TE method n whch only two varables x and y are consdered, the causalty between x and y cannot be detected, and then we conclude that there s no causal relatonshp between x and y, whch s consstent wth the fact. 2 For calculatons of the tradtonal TE, the partal TE, and the DTE, t s mportant to determne the parameters for each varable,.e., the predcton horzon, embeddng dmensons, and the tme nterval. Much research has been done on how to determne these parameters for the tradtonal TE. For example, n [17], many smulatons have been done to determne these parameters. In ths paper, we also propose a parameter determnaton method for calculatng the tradtonal TE n Secton II C. We can see that, for only two varables, t s not easy to determne the parameters. If all the envronmental varables are consdered as ntermedate varables, the parameters for a large number of varables need to be chosen smultaneously and approprately, whch s nontrval to acheve. But, for the DTE method, the parameters are determned based on the calculaton of the tradtonal transfer entropes; thus, as long as the parameters of the tradtonal TE for each par of the varables are determned, the parameters n the formulaton of the DTE can be determned accordngly. Detals can be found n the defnton of the DTE n Secton IIA. 3 From the applcaton pont of vew, the utlty of the partal TE s to detect undrectonal causaltes [25]. The authors of [25] use the dfference between the partal TE from x to y and the partal TE from y to x to quantfy the causalty from x to y, whch s sutable n neuroscences; however, n ndustral processes, feedback and bdrectonal causaltes are common due to recycle streams. If we stll use the dfference between TE from x to y and the TE from y to x to quantfy the causalty from x to y, t s most lkely to lead to the concluson that there s no causal relatonshp between x and y, although n fact there s bdrectonal causalty between x and y. Thus, we use the calculated TE and DTE to quantfy total causalty and drect causalty, respectvely. In addton, we propose normalzaton methods to quantfy the strength of the total causalty and drect causalty, respectvely. III. EXAMPLES In ths secton, we gve three examples to show the usefulness of the proposed method. The frst two examples use smple mathematcal equatons to represent causal relatonshps and the thrd example s a smulated 2 2 multplenput multpleoutput (MIMO system. Example 1: Assume three lnear correlated contnuous random varables x, y, andz satsfyng { zk+1 = 0.8x k + 0.2z k + v 1k y k+1 = 0.6z k + v 2k where x k N(0, 1, v 1k,v 2k N(0, 0.1, andz(0 = 3.2. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. To calculate the transfer entropes between x, z, and y, we need to determne the four desgn parameters. We take the TE from x to y n (1 as an example. Frst, we set ntal values for h 1 and τ 1 as h 1 = τ 1 = 1. Second, we calculate H c (y +h1 y (k 1 wth k 1 = 0, 1,...,10, as shown n the upper part of Fg. 4. The change rate of H c (y +h1 y (k 1 wth k 1 = 0, 1,...,10 s shown n the lower part of Fg. 4; we can see that as k 1 ncreases, the change rate of H c (y +h1 y (k 1 does not vary sharply, whch means that the hstory of y does not provde useful nformaton for the future values of y. Therefore, we choose k 1 = 0. Fnally, we calculate the TE T x y and ts change rate wth l 1 = 1,...,10, as shown n Fg. 5. Snce the change rate of T x y decreases sgnfcantly after l 1 = 2, as shown n the lower part of Fg. 5, we choose l 1 = 2. Usng the same procedure, the parameters for each par of x, z, andy are determned as h 1 = h 2 = h 3 = 1, τ 1 = τ 2 = τ 3 = 1, k 1 = m 1 = k 2 = 0, l 1 = 2, and l 2 = m 2 = 1. For the followng example and case studes, the same procedure s used. After the parameters are determned accordng to (19, the normalzed transfer entropes between each par of x, y, andz are shown n Table I. We can see that x causes z, z causes y, and x causes y because NTE c x z = 0.409, NTEc z y = 0.393, and NTE c x y = are relatvely large. Thus we need to frst determne whether there s drect causalty from x to y. Accordng to (4, we obtan D x y = Accordng to (20, the normalzed DTE from x to y s NDTE c x y = 0.016, whch s very small. Thus, we conclude that there s almost no drect causalty from x to y. The nformaton flow pathways for Example 1 are shown n Fg. 6(a. Ths concluson s consstent wth the mathematcal functon, from whch we can see that the nformaton flow from x to y s through the ntermedate varable z and there s no drect nformaton flow pathway from x to y. Example 2: Assume three nonlnear correlated contnuous random varables x, y, andz satsfyng { zk+1 = (0.8x k z k +v 1k y k+1 = 5(z k x k +v 2k
10 DUAN et al.: DIRECT CAUSALITY DETECTION VIA TE y( k 1 y H ( c Fg. 6. Informaton flow pathways for (a Example 1. (b Example 2. 0 k 1 TABLE II NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 2 +1 y( k 1 ΔH (y c k 1 NTE c row column x z y x NA z 0 NA y NA Fg. 4. Fndng the embeddng dmenson of y for Example T x y l 1 Fg. 7. System block dagram for Example 3. Δ T x y l 1 Fg. 5. Fndng the embeddng dmenson of x for T x y of Example 1. TABLE I NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 1 NTE c row column x z y x NA z NA y NA where x k [4, 5] s a unform dstrbuted sgnal, v 1k,v 2k N(0, 0.05, and z(0 = 0.2. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. The normalzed transfer entropes between each par of x, z, andy are shown n Table II. We can see that x causes z, z causes y, andx causes y because NTE c x z = 0.623, NTE c z y = 0.308, and NTE c x y = are relatvely large. Thus, we need to frst determne whether there s drect causalty from x to y. Accordng to (4, we obtan D x y = Accordng to (20, the normalzed DTE from x to y s NDTE c x y = 0.304, whch s much larger than zero. Thus, we conclude that there s drect causalty from x to y. Second, we need to detect whether there s true and drect causalty from z to y. Accordng to (5, we obtan D z y = 0.538, and thus the normalzed DTE from z to y s NDTE c z y = 0.438, whch s much larger than zero. Hence, we conclude that there s true and drect causalty from z to y. The nformaton flow pathways for Example 2 are shown n Fg. 6(b. Ths concluson s consstent wth the mathematcal functon, from whch we can see that there are drect nformaton flow pathways both from x to y and from z to y. Example 3: Fg. 7 shows a block dagram of a MIMO system wth two nputs r 1 and r 2, and two outputs y 1 and y 2. Assume that r 1 N(0, 1 and r 2 N(0, 1 are ndependent, and v N(0, 0.1 s the sensor nose. The smulaton data conssts of 6000 samples. To ensure statonarty, the ntal 3000 data ponts were dscarded. The normalzed transfer entropes between each par of r 1, r 2, y 1,andy 2 are shown n Table III. We can see that r 1 causes y 1 and y 2, r 2 also cause y 1 and y 2,andy 2 causes y 1. The correspondng nformaton flow pathways are shown n Fg. 8.
11 2062 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 6, NOVEMBER 2013 TABLE III NORMALIZED TRANSFER ENTROPIES FOR EXAMPLE 3 NTE c row column r 1 r 2 y 1 y 2 r 1 NA r NA y NA y NA Fg. 8. Informaton flow pathways for Example 3. AsshownnFg.8,sncey 1 and y 2 have common sources r 1 and r 2, we need to frst detect whether the causalty from y 2 to y 1 s true or spurous. Accordng to (21, we obtan that the DTE from y 2 to y 1 wth ntermedate varables r 1 and r 2 s D y2 y 1 = Accordng to (20, the normalzed DTE from y 2 to y 1 s NDTE c y 2 y 1 = 0.366, whch s much larger than zero. Hence, we conclude that there s true and drect causalty from y 2 to y 1. Second, snce r 1 causes y 2, y 2 causes y 1,andr 1 causes y 1, we need to further detect whether there s drect causalty from r 1 to y 1. Accordng to (4, we obtan that the DTE from r 1 to y 1 wth the ntermedate varable y 2 s D r1 y 1 = Accordng to (20, the normalzed DTE from r 1 to y 1 s NDTEr c 1 y 1 = 0.573, whch s much larger than zero. Thus, we conclude that there s drect causalty from r 1 to y 1 n addton to the ndrect causalty through ntermedate varable y 2. Smlarly, we obtan that the DTE from r 2 to y 1 wth the ntermedate varable y 2 s D r2 y 1 = and the normalzed DTE from r 2 to y 1 s NDTEr c 2 y 1 = 0.617, whch s also much larger than zero. Thus, we conclude that there s drect causalty from r 2 to y 1. The nformaton flow pathways are the same as those obtaned from the results of calculated TEs, as shown n Fg. 8. Ths concluson s consstent wth the block dagram, from whch we can see that there are drect nformaton flow pathways from r 1 to y 1, from r 2 to y 1, and from y 2 to y 1. No matter whether the relatonshps of varables are lnear or nonlnear, the DTE can detect drect causalty and the normalzed DTE can quantfy the strength of drect causalty. IV. CASE STUDIES In ths secton, an expermental and an ndustral case studes are llustrated to valdate the proposed drect causalty detecton method. A. Expermental Case Study In order to show the effectveness of the proposed methods, a threetank experment was conducted. The schematc of the threetank system s shown n Fg. 9. Water s drawn from a Fg. 9. Fg. 10. Schematc of the threetank system. x 4 x 3 x 2 x 1 Tme Trends Samples Tme trends of measurements of the threetank system. reservor and pumped to tanks 1 and 2 by a gear pump and a three way valve. The water n tank 2 can flow down nto tank 3. The water n tanks 1 and 3 eventually flows down nto the reservor. The experment s conducted under openloop condtons. The water levels are measured by level transmtters. We denote the water levels of tanks 1 3 by x 1, x 2, and x 3, respectvely. The flow rate of the water out of the pump s measured by a flow meter; we denote ths flow rate by x 4.In ths experment, the normal flow rate of the water out of the pump s 10 l/mn. However, the flow rate vares randomly wth a mean value of 10 l/mn because of the nose n the sensor and mnor fluctuatons n the pump. The sampled data of 3000 observatons are analyzed. Fg. 10 shows the normalzed tme trends of the measurements. The samplng tme s 1 s. In order to detect the causalty and drect causalty usng TE and DTE, we need to frst test the statonarty of the dataset. Takng x 1 as an example, we dvde the 3000 data ponts nto
Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting
Causal, Explanatory Forecastng Assumes causeandeffect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationx f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60
BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true
More informationTHE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES
The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationQuantization Effects in Digital Filters
Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value
More informationTime Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University
Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton
More informationPSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 12
14 The Chsquared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationSIMPLE LINEAR CORRELATION
SIMPLE LINEAR CORRELATION Smple lnear correlaton s a measure of the degree to whch two varables vary together, or a measure of the ntensty of the assocaton between two varables. Correlaton often s abused.
More informationThe covariance is the two variable analog to the variance. The formula for the covariance between two variables is
Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.
More informationRecurrence. 1 Definitions and main statements
Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.
More informationRESEARCH ON DUALSHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo.
ICSV4 Carns Australa 9 July, 007 RESEARCH ON DUALSHAKER SINE VIBRATION CONTROL Yaoq FENG, Hanpng QIU Dynamc Test Laboratory, BISEE Chna Academy of Space Technology (CAST) yaoq.feng@yahoo.com Abstract
More informationRiskbased Fatigue Estimate of Deep Water Risers  Course Project for EM388F: Fracture Mechanics, Spring 2008
Rskbased Fatgue Estmate of Deep Water Rsers  Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More informationCan Auto Liability Insurance Purchases Signal Risk Attitude?
Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? ChuShu L Department of Internatonal Busness, Asa Unversty, Tawan ShengChang
More information8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by
6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng
More informationCHOLESTEROL REFERENCE METHOD LABORATORY NETWORK. Sample Stability Protocol
CHOLESTEROL REFERENCE METHOD LABORATORY NETWORK Sample Stablty Protocol Background The Cholesterol Reference Method Laboratory Network (CRMLN) developed certfcaton protocols for total cholesterol, HDL
More information2.4 Bivariate distributions
page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together
More informationLuby s Alg. for Maximal Independent Sets using Pairwise Independence
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
More informationCalculation of Sampling Weights
Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a twostage stratfed cluster desgn. 1 The frst stage conssted of a sample
More informationThe Analysis of Covariance. ERSH 8310 Keppel and Wickens Chapter 15
The Analyss of Covarance ERSH 830 Keppel and Wckens Chapter 5 Today s Class Intal Consderatons Covarance and Lnear Regresson The Lnear Regresson Equaton TheAnalyss of Covarance Assumptons Underlyng the
More information1. Measuring association using correlation and regression
How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING
ANALYZING THE RELATIONSHIPS BETWEEN QUALITY, TIME, AND COST IN PROJECT MANAGEMENT DECISION MAKING Matthew J. Lberatore, Department of Management and Operatons, Vllanova Unversty, Vllanova, PA 19085, 6105194390,
More informationThe Development of Web Log Mining Based on ImproveKMeans Clustering Analysis
The Development of Web Log Mnng Based on ImproveKMeans Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationInequality and The Accounting Period. Quentin Wodon and Shlomo Yitzhaki. World Bank and Hebrew University. September 2001.
Inequalty and The Accountng Perod Quentn Wodon and Shlomo Ytzha World Ban and Hebrew Unversty September Abstract Income nequalty typcally declnes wth the length of tme taen nto account for measurement.
More informationThe Analysis of Outliers in Statistical Data
THALES Project No. xxxx The Analyss of Outlers n Statstcal Data Research Team Chrysses Caron, Assocate Professor (P.I.) Vaslk Karot, Doctoral canddate Polychrons Economou, Chrstna Perrakou, Postgraduate
More informationErrorPropagation.nb 1. Error Propagation
ErrorPropagaton.nb Error Propagaton Suppose that we make observatons of a quantty x that s subject to random fluctuatons or measurement errors. Our best estmate of the true value for ths quantty s then
More informationAn Alternative Way to Measure Private Equity Performance
An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate
More informationBERNSTEIN POLYNOMIALS
OnLne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful
More informationDamage detection in composite laminates using cointap method
Damage detecton n composte lamnates usng contap method S.J. Km Korea Aerospace Research Insttute, 45 EoeunDong, YouseongGu, 35333 Daejeon, Republc of Korea yaeln@kar.re.kr 45 The contap test has the
More informationStudy on CET4 Marks in China s Graded English Teaching
Study on CET4 Marks n Chna s Graded Englsh Teachng CHE We College of Foregn Studes, Shandong Insttute of Busness and Technology, P.R.Chna, 264005 Abstract: Ths paper deploys Logt model, and decomposes
More informationThe OC Curve of Attribute Acceptance Plans
The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4
More informationAn Evaluation of the Extended Logistic, Simple Logistic, and Gompertz Models for Forecasting Short Lifecycle Products and Services
An Evaluaton of the Extended Logstc, Smple Logstc, and Gompertz Models for Forecastng Short Lfecycle Products and Servces Charles V. Trappey a,1, Hsnyng Wu b a Professor (Management Scence), Natonal Chao
More informationCHAPTER 14 MORE ABOUT REGRESSION
CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp
More informationThe Application of Fractional Brownian Motion in Option Pricing
Vol. 0, No. (05), pp. 738 http://dx.do.org/0.457/jmue.05.0..6 The Applcaton of Fractonal Brownan Moton n Opton Prcng Qngxn Zhou School of Basc Scence,arbn Unversty of Commerce,arbn zhouqngxn98@6.com
More informationA Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy Scurve Regression
Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy Scurve Regresson ChengWu Chen, Morrs H. L. Wang and TngYa Hseh Department of Cvl Engneerng, Natonal Central Unversty,
More information1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)
6.3 /  Communcaton Networks II (Görg) SS20  www.comnets.unbremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes
More informationSPEE Recommended Evaluation Practice #6 Definition of Decline Curve Parameters Background:
SPEE Recommended Evaluaton Practce #6 efnton of eclne Curve Parameters Background: The producton hstores of ol and gas wells can be analyzed to estmate reserves and future ol and gas producton rates and
More informationTHE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek
HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo
More information"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *
Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC
More information1 Example 1: Axisaligned rectangles
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton
More informationPowerofTwo Policies for Single Warehouse MultiRetailer Inventory Systems with Order Frequency Discounts
Powerofwo Polces for Sngle Warehouse MultRetaler Inventory Systems wth Order Frequency Dscounts José A. Ventura Pennsylvana State Unversty (USA) Yale. Herer echnon Israel Insttute of echnology (Israel)
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationFormula of Total Probability, Bayes Rule, and Applications
1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.
More informationNPAR TESTS. OneSample ChiSquare Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6
PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has
More informationBrigid Mullany, Ph.D University of North Carolina, Charlotte
Evaluaton And Comparson Of The Dfferent Standards Used To Defne The Postonal Accuracy And Repeatablty Of Numercally Controlled Machnng Center Axes Brgd Mullany, Ph.D Unversty of North Carolna, Charlotte
More informationRing structure of splines on triangulations
www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAMReport 201448 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon
More informationPassive Filters. References: Barbow (pp 265275), Hayes & Horowitz (pp 3260), Rizzoni (Chap. 6)
Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called
More information1 Approximation Algorithms
CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons
More informationStatistical Methods to Develop Rating Models
Statstcal Methods to Develop Ratng Models [Evelyn Hayden and Danel Porath, Österrechsche Natonalbank and Unversty of Appled Scences at Manz] Source: The Basel II Rsk Parameters Estmaton, Valdaton, and
More informationCredit Limit Optimization (CLO) for Credit Cards
Credt Lmt Optmzaton (CLO) for Credt Cards Vay S. Desa CSCC IX, Ednburgh September 8, 2005 Copyrght 2003, SAS Insttute Inc. All rghts reserved. SAS Propretary Agenda Background Tradtonal approaches to credt
More informationProject Networks With MixedTime Constraints
Project Networs Wth MxedTme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa
More informationExhaustive Regression. An Exploration of RegressionBased Data Mining Techniques Using Super Computation
Exhaustve Regresson An Exploraton of RegressonBased Data Mnng Technques Usng Super Computaton Antony Daves, Ph.D. Assocate Professor of Economcs Duquesne Unversty Pttsburgh, PA 58 Research Fellow The
More informationImplementation of Deutsch's Algorithm Using Mathcad
Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages  n "Machnes, Logc and Quantum Physcs"
More informationgreatest common divisor
4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no
More informationA hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm
Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):18841889 Research Artcle ISSN : 09757384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel
More informationOn the Optimal Control of a Cascade of HydroElectric Power Stations
On the Optmal Control of a Cascade of HydroElectrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;
More informationCharacterization of Assembly. Variation Analysis Methods. A Thesis. Presented to the. Department of Mechanical Engineering. Brigham Young University
Characterzaton of Assembly Varaton Analyss Methods A Thess Presented to the Department of Mechancal Engneerng Brgham Young Unversty In Partal Fulfllment of the Requrements for the Degree Master of Scence
More informationDEFINING %COMPLETE IN MICROSOFT PROJECT
CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMISP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,
More informationTime Domain simulation of PD Propagation in XLPE Cables Considering Frequency Dependent Parameters
Internatonal Journal of Smart Grd and Clean Energy Tme Doman smulaton of PD Propagaton n XLPE Cables Consderng Frequency Dependent Parameters We Zhang a, Jan He b, Ln Tan b, Xuejun Lv b, HongJe L a *
More informationControl Charts for Means (Simulation)
Chapter 290 Control Charts for Means (Smulaton) Introducton Ths procedure allows you to study the run length dstrbuton of Shewhart (Xbar), Cusum, FIR Cusum, and EWMA process control charts for means usng
More informationv a 1 b 1 i, a 2 b 2 i,..., a n b n i.
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationData Broadcast on a MultiSystem Heterogeneous Overlayed Wireless Network *
JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, 819840 (2008) Data Broadcast on a MultSystem Heterogeneous Overlayed Wreless Network * Department of Computer Scence Natonal Chao Tung Unversty Hsnchu,
More informationSTATISTICAL DATA ANALYSIS IN EXCEL
Mcroarray Center STATISTICAL DATA ANALYSIS IN EXCEL Lecture 6 Some Advanced Topcs Dr. Petr Nazarov 1401013 petr.nazarov@crpsante.lu Statstcal data analyss n Ecel. 6. Some advanced topcs Correcton for
More informationAn InterestOriented Network Evolution Mechanism for Online Communities
An InterestOrented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationIMPACT ANALYSIS OF A CELLULAR PHONE
4 th ASA & μeta Internatonal Conference IMPACT AALYSIS OF A CELLULAR PHOE We Lu, 2 Hongy L Bejng FEAonlne Engneerng Co.,Ltd. Bejng, Chna ABSTRACT Drop test smulaton plays an mportant role n nvestgatng
More informationAnalysis of Covariance
Chapter 551 Analyss of Covarance Introducton A common tas n research s to compare the averages of two or more populatons (groups). We mght want to compare the ncome level of two regons, the ntrogen content
More informationOptimal Bidding Strategies for Generation Companies in a DayAhead Electricity Market with Risk Management Taken into Account
Amercan J. of Engneerng and Appled Scences (): 86, 009 ISSN 94700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a DayAhead Electrcty Market wth Rsk Management Taken nto Account
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationInstitute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic
Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange
More informationFrequency Selective IQ Phase and IQ Amplitude Imbalance Adjustments for OFDM Direct Conversion Transmitters
Frequency Selectve IQ Phase and IQ Ampltude Imbalance Adjustments for OFDM Drect Converson ransmtters Edmund Coersmeer, Ernst Zelnsk Noka, Meesmannstrasse 103, 44807 Bochum, Germany edmund.coersmeer@noka.com,
More informationScalar and Vector Quantization
Scalar and Vector Quantzaton Máro A. T. Fgueredo, Departamento de Engenhara Electrotécnca e de Computadores, Insttuto Superor Técnco, Lsboa, Portugal maro.fgueredo@st.utl.pt November 2008 Quantzaton s
More informationGRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 NORM
GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 NORM BARRIOT JeanPerre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jeanperre.barrot@cnes.fr 1/Introducton The
More informationFORCED CONVECTION HEAT TRANSFER IN A DOUBLE PIPE HEAT EXCHANGER
FORCED CONVECION HEA RANSFER IN A DOUBLE PIPE HEA EXCHANGER Dr. J. Mchael Doster Department of Nuclear Engneerng Box 7909 North Carolna State Unversty Ralegh, NC 276957909 Introducton he convectve heat
More informationAnswer: A). There is a flatter IS curve in the high MPC economy. Original LM LM after increase in M. IS curve for low MPC economy
4.02 Quz Solutons Fall 2004 MultpleChoce Questons (30/00 ponts) Please, crcle the correct answer for each of the followng 0 multplechoce questons. For each queston, only one of the answers s correct.
More informationCHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES
CHAPTER 5 RELATIONSHIPS BETWEEN QUANTITATIVE VARIABLES In ths chapter, we wll learn how to descrbe the relatonshp between two quanttatve varables. Remember (from Chapter 2) that the terms quanttatve varable
More informationToday s class. Chapter 13. Sources of uncertainty. Decision making with uncertainty
Today s class Probablty theory Bayesan nference From the ont dstrbuton Usng ndependence/factorng From sources of evdence Chapter 13 1 2 Sources of uncertanty Uncertan nputs Mssng data Nosy data Uncertan
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationMAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date
Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPPATBDClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller
More informationJ. Parallel Distrib. Comput.
J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n
More informationINVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMAHDR NETWORKS
21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS
More informationLecture 3: Force of Interest, Real Interest Rate, Annuity
Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annutymmedate, and ts present value Study annutydue, and
More informationStudy on Model of Risks Assessment of Standard Operation in Rural Power Network
Study on Model of Rsks Assessment of Standard Operaton n Rural Power Network Qngj L 1, Tao Yang 2 1 Qngj L, College of Informaton and Electrcal Engneerng, Shenyang Agrculture Unversty, Shenyang 110866,
More informationSimulation and optimization of supply chains: alternative or complementary approaches?
Smulaton and optmzaton of supply chans: alternatve or complementary approaches? Chrstan Almeder Margaretha Preusser Rchard F. Hartl Orgnally publshed n: OR Spectrum (2009) 31:95 119 DOI 10.1007/s002910070118z
More informationNew bounds in BalogSzemerédiGowers theorem
New bounds n BalogSzemerédGowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A
More informationProperties of Indoor Received Signal Strength for WLAN Location Fingerprinting
Propertes of Indoor Receved Sgnal Strength for WLAN Locaton Fngerprntng Kamol Kaemarungs and Prashant Krshnamurthy Telecommuncatons Program, School of Informaton Scences, Unversty of Pttsburgh Emal: kakst2,prashk@ptt.edu
More informationAn Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems
STANCS73355 I SUSE73013 An Analyss of Central Processor Schedulng n Multprogrammed Computer Systems (Dgest Edton) by Thomas G. Prce October 1972 Techncal Report No. 57 Reproducton n whole or n part
More information+ + +   This circuit than can be reduced to a planar circuit
MeshCurrent Method The meshcurrent s analog of the nodeoltage method. We sole for a new set of arables, mesh currents, that automatcally satsfy KCLs. As such, meshcurrent method reduces crcut soluton to
More informationDECOMPOSING ALLOCATIVE EFFICIENCY FOR MULTIPRODUCT PRODUCTION SYSTEMS
DECOMPOSING ALLOCATIVE EFFICIENCY FOR MULTIPRODUCT PRODUCTION SYSTEMS EKONOMIKA A MANAGEMENT Tao Zhang Introducton Data envelopment analyss (DEA, the nonparametrc approach to measurng effcency, was frst
More informationRisk Model of LongTerm Production Scheduling in Open Pit Gold Mining
Rsk Model of LongTerm Producton Schedulng n Open Pt Gold Mnng R Halatchev 1 and P Lever 2 ABSTRACT Open pt gold mnng s an mportant sector of the Australan mnng ndustry. It uses large amounts of nvestments,
More informationConversion between the vector and raster data structures using Fuzzy Geographical Entities
Converson between the vector and raster data structures usng Fuzzy Geographcal Enttes Cdála Fonte Department of Mathematcs Faculty of Scences and Technology Unversty of Combra, Apartado 38, 3 454 Combra,
More information7.5. Present Value of an Annuity. Investigate
7.5 Present Value of an Annuty Owen and Anna are approachng retrement and are puttng ther fnances n order. They have worked hard and nvested ther earnngs so that they now have a large amount of money on
More informationAn MILP model for planning of batch plants operating in a campaignmode
An MILP model for plannng of batch plants operatng n a campagnmode Yanna Fumero Insttuto de Desarrollo y Dseño CONICET UTN yfumero@santafeconcet.gov.ar Gabrela Corsano Insttuto de Desarrollo y Dseño
More informationStochastic Protocol Modeling for Anomaly Based Network Intrusion Detection
Stochastc Protocol Modelng for Anomaly Based Network Intruson Detecton Juan M. EstevezTapador, Pedro GarcaTeodoro, and Jesus E. DazVerdejo Department of Electroncs and Computer Technology Unversty of
More informationPRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIGIOUS AFFILIATION AND PARTICIPATION
PRIVATE SCHOOL CHOICE: THE EFFECTS OF RELIIOUS AFFILIATION AND PARTICIPATION Danny CohenZada Department of Economcs, Benuron Unversty, BeerSheva 84105, Israel Wllam Sander Department of Economcs, DePaul
More informationPortfolio Loss Distribution
Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets holdtomaturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment
More information