Gene Regulatory Network Discovery from Time-Series Gene Expression Data A Computational Intelligence Approach

Transcription

1 Gene Regulaory Nework Discovery from Time-Series Gene Expression Daa A Compuaional Inelligence Approach Nikola K. Kasabov 1, Zeke S. H. Chan 1, Vishal Jain 1, Igor Sidorov 2 and Dimier S. Dimirov 2 1 Knowledge Engineering and Discovery Research Insiue (KEDRI), Auckland Universiy of Technology, Privae Bag 92006, Auckland, New Zealand {nkasabov, shchan, vishal.jain}@au.ac.nz 2 Naional Cancer Insiue, Frederick, Washingon DC, Naional Insiue of Healh, USA {sidorovi, dimirov}@ncifcrf.gov Absrac. The inerplay of ineracions beween DNA, RNA and proeins leads o geneic regulaory neworks (GRN) and in urn conrols he gene regulaion. Direcly or indirecly in a cell such molecules eiher inerac in a posiive or in repressive manner herefore i is hard o obain he accurae compuaional models hrough which he final sae of a cell can be prediced wih cerain accuracy. This paper describes biological behaviour of acual regulaory sysems and we propose a novel mehod for GRN discovery of a large number of genes from muliple ime series gene expression observaions over small and irregular ime inervals. The mehod inegraes a geneic algorihm (GA) o selec a small number of genes and a Kalman filer o derive he GRN of hese genes. Afer GRNs of smaller number of genes are obained, hese GRNs may be inegraed in order o creae he GRN of a larger group of genes of ineres. 1 Inroducion Gene regulaory nework is one of he wo main arges in biological sysems because hey are sysems conrolling he fundamenal mechanisms ha govern biological sysems. A single gene ineracs wih many oher genes in he cell, inhibiing or promoing direcly or indirecly, he expression of some of hem a he same ime. Gene ineracion may conrol wheher and how vigorously ha gene will produce RNA wih he help of a group of imporan proeins known as ranscripion facors. When hese acive ranscripion facors associae wih he arge gene sequence (DNA bases), hey can funcion o specifically suppress or acivae synhesis of he corresponding RNA. Each RNA ranscrip hen funcions as he emplae for synhesis of a specific proein. Thus he gene, ranscripion facor and oher proeins may inerac in a manner ha is very imporan for deerminaion of cell funcion. Much less is known abou he funcioning of he regulaory sysems of which he individual genes and ineracion form a par [6], [8], [15], [20]. Transcripion facors provide a feedback pahway by which genes can regulae one anoher s expression as mrna and hen as proein [3], [5].

2 2 Nikola K. Kasabov1, Zeke S. H. Chan1, Vishal Jain1, Igor Sidorov2 and Dimier S. Dimirov2 The discovery of gene regulaory neworks (GRN) from ime series of gene expression observaions can be used o: (1) Idenify imporan genes in relaion o a disease or a biological funcion, (2) Gain an undersanding on he dynamic ineracion beween genes, (3) Predic gene expression values a fuure ime poins. The major approaches ha deals wih he modelling of gene regulaory neworks involve differenial equaions [14], sochasic models [16], evolving connecionis sysems [13], boolean neworks [18], generalized logical equaions [21], hreshold models [19], peri nes [11], bayesian neworks [9], direced and undireced graphs. We propose here a novel mehod ha inegraes Kalman Filer [4] and Geneic Algorihm (GA) [10], [12]. The GA is used o selec a small number of genes, and he Kalman filer mehod is used o derive he GRN of hese genes. Afer GRNs of smaller number of genes are obained, hese GRNs may be inegraed in order o creae he GRN of a larger group of genes of ineres. The goal of his work is develop a mehod for GRN discovery from muliple and shor ime series daa of a large number of genes. The secondary goal is o apply he mehod as o idenify he genes ha co-regulae elomerase from he exracs of he U937 plus and minus series obained in NCI, NIH. Each series conains he ime-series expression of 32 preseleced candidae genes ha have been found poenially relevan, as well as he expression of he elomerase. Boh he plus series and minus series conains four samples recorded a he (0, 6, 24, 48) h hour. Discovering GRN from hese wo series is challenging in wo aspecs: firs, boh series are sampled a irregular ime inervals; second, he number of samples is scarce (only 4 samples). A hird poenial problem is ha he search space grows exponenially in size as more candidae genes are idenified in he fuure. Several GRNs of 3 mos relaed o he elomerase genes are discovered, analysed and inegraed. The resuls and heir inerpreaion confirm he validiy and he applicabiliy of he proposed mehod. The inegraed mehod can be easily generalized o exrac GRN from oher ime series gene expression daa. This paper repors he mehodology and he experimenal findings. 2 Modelling GRN wih firs-order differenial equaions, saespace represenaion and Kalman Filer 2.1 Discree-Time Approximaion of Firs-Order Differenial Equaions Our GRN is modelled wih he discree ime approximaion of firs-order differenial equaions, given by: x + +1 = Fx (1) where x = ( x, x2,... xn ) is he gene expression a he -h ime inerval and n is he number of genes modelled, is a noise componen wih covariance E=cov ( ), and F=(f ij ) i=1,n, j=1,n is he ransiion marix relaing x o x +1. I is relaed o he coninuous firs-order differenial equaions d x d = x + e by F = τ + I and = τe 1

3 Gene Regulaory Nework Discovery from Time-Series Gene Expression Daa A Compuaional Inelligence Approach 3 where is he ime inerval {noe he subscrip noaion (+k) is acually he common abbreviaion for (+k )}[7]. We work here wih a discree approximaion insead of a coninuous model for he ease of modelling and processing he irregular ime course daa (wih Kalman filer). Besides being a ool widely used for modelling biological processes, here are wo advanages in using firs-order differenial equaions. Firs, gene relaions can be elucidaed from he ransiion marix F hrough choosing a hreshold value (ζ; 1>ζ>0). If f ij is larger hen he hreshold value ζ, x, j is assumed o have significan influence on x +1,i. A posiive value of f ij indicaes a posiive influence and vice-versa. Second, hey can be easily manipulaed wih KF o handle irregularly sampled daa, which allow parameer esimaion, likelihood evaluaion and model simulaion and predicion. The main drawback of using differenial equaions is ha i requires he esimaion of n 2 parameers for he ransiion marix F and n (n-1)/2 parameers for he noise covariance E. To minimize he number of model parameers, we esimae only F and fix E o a small value. Since boh series conain only 4 samples, we avoid overparameerizaion by seing n o 4, which is he maximum number of n before he number of parameers exceeds he number of raining daa {I maches he number of model parameers (he size of F is n 2 =16) o he number of raining daa (n 4 samples =16)}. Since in our case sudy one of he n genes mus be elomerase, we can search for a subse of size K=3 oher genes o form a GRN. To handle irregularly sampled daa, we employ he sae-space mehodology and he KF. We rea he rue rajecories as a se of unobserved or hidden variables called he sae variables, and hen apply he KF o compue heir opimal esimaes based on he observed daa. The sae variables ha are regular/complee can now be applied o perform model funcions like predicion, parameer esimaions insead of he observed daa ha are irregular/incomplee. This approach is more superior o inerpolaion mehods as i prevens false modelling by rusing a fixed se of inerpolaed poins ha may be erroneous. 2.2 Sae-Space Represenaion To apply he sae-space mehodology, a model mus be expressed in he following forma called he discree-ime sae space represenaion x = x + w +1 (2) y = Ax + v (3) cov( v ) = R cov( w ) = Q (4) where, x is he sysem sae; y is he observed daa; Φ is he sae ransiion marix ha relaes x o x +1; A is he linear connecion marix ha relaes x o y ; w and v are uncorrelaed whie noise sequences whose covariance marices are Q and R respecively. The firs equaion is called he sae equaion ha describes he

4 4 Nikola K. Kasabov1, Zeke S. H. Chan1, Vishal Jain1, Igor Sidorov2 and Dimier S. Dimirov2 dynamics of he sae variables. The second equaion is called he observaion equaion ha relaes he saes o he observaion. To represen he discree-ime model in he sae-space forma, we simply subsiue he discree-ime equaion (1) ino he sae equaion (2) by seing Φ=F, w =ε and Q=E and form a direc mapping beween saes and observaions by seing A=I. The sae ransiion marix Φ (funcional equivalen o F) is he parameer of ineres as i relaes he fuure response of he sysem o he presen sae and governs he dynamics of he enire sysem. The covariance marices Q and R are of secondary ineres and are fixed o small values o reduce he number of model parameers. 2.3 Kalman Filer (KF) KF is a se of recursive equaions capable of compuing opimal esimaes (in he leas-square sense) of he pas, presen and fuure saes of he sae-space model based on he observed daa. Here we use i o esimae gene expression rajecories given irregularly sampled daa. To specify he operaion of Kalman filer, we define he condiional mean value of he sae x s and is covariance P u s as: s x = E( x y, y 2,..., y 1 s s s [ x x )( x x ) y ] P,..., s u = E ( u u 1 y s For predicion, we use he KF forward recursions o compue he sae esimaes for (s<). For likelihood evaluaion and parameer esimaion, we use he KF backward recursions o compue he esimaes called he smoohed esimaes based on he enire daa, i.e. (s=t; T> is he index of he las observaion), which in urn are used o compue he required saisics. ) (5) (6) 2.4 Using GA for he selecion of a gene subse for a GRN The ask is o search for he genes ha form he mos probable GRN models, using he model likelihood compued by he KF as an objecive funcion. Given N he number of candidaes and K he size of he subse, he number of differen gene combinaions is N!/K!(N-K)!. In our case sudy, N=32 is small enough for an exhausive search. However, as more candidaes are idenified in he fuure, he search space grows exponenially in size and exhausive search will soon become infeasible. For his reason a mehod based on GA is proposed. The srengh of GA is wofold: 1. Unlike mos classical gradien mehods or greedy algorihms ha search along a single hill-climbing pah, a GA searches wih muliple poins and generaes new poins hrough applying geneic operaors ha are sochasic in naure. These properies allow for he search o escape local opima in a muli-modal environmen. GA is herefore useful for opimizing high dimensional funcions and noisy funcions whose search space conains many local opima poins.

5 Gene Regulaory Nework Discovery from Time-Series Gene Expression Daa A Compuaional Inelligence Approach 5 2. A GA is more effecive han a random search mehod as i focuses is search in he promising regions of he problem space. 2.5 GA Design for Gene Subse Selecion In he GA-based mehod for gene subse selecion proposed here, each soluion is coded as a binary sring of N bis. A 1 in he ih bi posiion denoes ha he ih gene is seleced and a 0 oherwise. Each soluion mus have exacly K 1 s and a repair operaor is included o add or delee 1 s when his is violaed. The geneic operaors used for crossover, muaion and selecion are respecively he sandard crossover, he binary muaion and he (µ, λ) selecion operaors. Since here are wo series he plus and he minus series of ime-course gene expression observaions in our case sudy, a new finess funcion is designed o incorporae he model likelihood in boh series. For each soluion, he ranking of is model likelihood in he plus series and in he minus series are obained and hen summed o obain a join finess ranking. This favors convergence owards soluions ha are consisenly good in boh he plus and he minus series. The approach is applicable o muliple ime series daa. 2.6 Procedures of he GA-based mehod for gene subse selecion Populaion Iniializaion. Creae a populaion of µ random individuals (genes from he iniial gene se, e.g. of 32) as he firs generaion parens. Reproducion. The goal of reproducion is o creae λ offspring from µ parens. This process involves hree seps: crossover, muaion and repair. Crossover. The crossover operaor ransfers parenal rais o he offspring. We use he uniform crossover ha samples he value of each bi posiion from he firs paren a he crossover probabiliy p c and from he second paren oherwise. In general, performance of GA is no sensiive o he crossover probabiliy and i is se o a large value in he range of [0.5, 0.9] [1]. Here we se i o 0.7. Muaion. The muaion operaor induces diversiy o he populaion by injecing new geneic maerial ino he offspring. For each bi posiion of he offspring, muaion invers he value a a small muaion rae p m. Performance of GA is very sensiive o he muaion probabiliy and i usually adaps a very small value o avoid disruping convergence. Here we use p m =1/N, which has been shown o be boh he lower bound value and he opimal value for many es funcions [17], [1], providing an average of one muaion in every offspring. Repair. The funcion of he repair operaor is o ensure ha each offspring soluion has exacly K 1 o presen he indices of he K seleced genes in he subse. If he number of 1 s is greaer han K, inver a 1 a random; and vice-versa. Repea he process unil he number of 1 s maches he subse size K.

6 6 Nikola K. Kasabov1, Zeke S. H. Chan1, Vishal Jain1, Igor Sidorov2 and Dimier S. Dimirov2 Finess Evaluaion. Here λ offspring individuals (soluions) are evaluaed for heir finess. For each offspring soluion, we obain he model likelihood in he boh he plus and he minus series and compue heir ranking (lower he rank, higher he likelihood) wihin he populaion. Nex, we sum he rankings and use he negaed sum as finess esimaion so ha he lower he join ranking, he higher he finess. Selecion. The selecion operaor deermines which offspring or parens will become he nex generaion parens based on heir finess funcion. We use he (µ, λ) scheme ha selecs he fies µ of λ offspring o be he nex generaion parens. I is worh comparing his scheme o anoher popular selecion scheme (µ+λ) ha selecs he fies µ of he join pool of µ parens and λ offspring o be he nex generaion parens, in which he bes-finess individuals found are always mainained in he populaion, convergence is herefore faser. We use he (µ, λ) scheme because i offers a slower bu more diversified search ha is less likely o be rapped in local opima. Tes for erminaion. Sop he procedure if he maximum number of generaions is reached. Oherwise go back o he reproducion phase. Upon compleion, GA reurns he highes likelihood GRNs found in boh he plus and he minus series of gene expression observaions. The proposed mehod includes running he GA-based procedure over many ieraions (e.g. 50) hus obaining differen GRN ha include possibly differen genes. Then we summarize he significance of he genes based on heir frequency of occurrence in hese GRNs and if necessary we pu ogeher all hese GRNs hus creaing a global GRN on he whole gene se. 3 Experimens and Resuls The inegraed GA-KF mehod inroduced above is applied o idenify genes ha regulae elomerase in a GRN from a se of 32 pre-seleced genes. Since he search space is small (only C 3 32 =4960 combinaions), we apply exhausive search as well as GA for validaion and comparaive analysis. The experimenal seings are as follows. The expression values of each gene in he plus and minus series are joinly normalized in he inerval [-1, 1]. The purpose of he join normalizaion is o preserve he informaion on he difference beween he wo series in he mean. For each subse of n genes defined by he GA, we apply KF for parameer esimaion and likelihood evaluaion of he GRN model. Each GRN is rained for a leas 50 epochs (which is usually sufficien) unil he likelihood value increases by less han 0.1. During raining, he model is esed for sabiliy by compuing he eigenvalues of (Φ-I) [2], [7]. If any of he real par of eigenvalues is posiive, he model is unsable and is abandoned. For he experimens repored in his paper a relaively low resource seings are used. Paren and offspring populaion sizes (µ, λ) are se o (20, 40) and maximum

7 Gene Regulaory Nework Discovery from Time-Series Gene Expression Daa A Compuaional Inelligence Approach 7 number of generaions is se o 50. These values are empirically found o yield consisen resuls over differen runs. We run i for 20 imes from differen iniial populaion o obain he cumulaed resuls. The resuls are inerpreed from he lis of 50 mos probable GRNs found in each series (we can lower his number o narrow down he shorlis of significan genes). The frequencies of each gene being par of he highes likelihood GRNs in he plus and in he minus series are recorded. Nex, a join frequency is calculaed by summing he wo frequencies. The genes ha have a high join frequency are considered o be significan in boh minus and plus series. For exhausive search, we simply run hrough all gene combinaions of 3 genes plus he elomerase; hen evolve hrough KF a GRN for each combinaion and record he likelihood of each model in boh he plus and minus series. A similar scoring sysem as GA s finess funcion is employed. We obain a join ranking by summing he model likelihood rankings in he plus series and he minus series, and hen coun he frequency of he genes ha belong o he bes 50 GRNs in he join ranking. The op en highes scoring genes obained by GA and exhausive search are abulaed in Table 1. Table 1. Significan genes exraced by GA and hrough an exhausive search from 32 seleced genes Rank Indices of significan genes found Indices of significan by GA (Freq. of occurrence in genes found by Minus GRNs, Freq. of occurrence exhausive search in Plus GRNs) and heir accession (gene Index) numbers in Genbank 1 27 (179,185) X M (261,0) U X (146, 48) J X (64, 118) X J (0, 159) M AL (118, 24) U X (0, 126) HG3523-HT D (111, 0) D U (0, 105) D HG3521-HT (75, 0) AL J04102 The resuls obained by GA and exhausive search are srikingly similar. In boh liss, seven ou of op en genes are common (genes 27, 12, 32, 20, 22, 5, 6) and four ou of op five genes are he same (genes 27, 12, 32 and 20). The similariy in he resuls suppors he applicabiliy of a GA-based mehod in his search problem and in paricular, when he search space is oo large for an exhausive search. An ousanding gene idenified is gene 27, TCF-1. The biological implicaions of TCF-1 and oher high scoring genes are currenly under invesigaion. The idenified GRNs can be used for model simulaion and predicion. The GRN dynamics can also be visualized wih a nework diagram using he influenial informaion exraced from he sae ransiion marix. As an example, we examine one of he discovered GRN of genes (33, 8, 27, 21) for boh he plus and minus series,

8 8 Nikola K. Kasabov1, Zeke S. H. Chan1, Vishal Jain1, Igor Sidorov2 and Dimier S. Dimirov2 shown in Fig. 1 and Fig. 2 respecively. The nework diagram shows only he componens of whose absolue values are above he hreshold value =0.3. For he plus series, he nework diagram in Fig. 1 (a) shows ha gene 27 has he mos significan role regulaing all oher genes (noe ha gene 27 has all is arrows ou-going). The nework simulaion, shown in Fig. 1 (b) fis he rue observaions well and he prediced values appear sable, suggesing ha he model is accurae and robus. For he minus series, he nework diagram in Fig. 2 (a) shows a differen nework from ha of he plus series. The role of gene 27 is no as prominen. The relaionship beween genes is no more causal bu inerdependen, wih genes 27, 33 and 21 simulaneously affecing each oher. The difference beween he plus and minus models is expeced. Again, he nework simulaion resul shown in Fig. 2 (b) shows ha he model fis he daa well and he predicion appears reasonable. predicions (a) Fig. 1. The idenified bes GRN of gene 33 (elomerase) and genes 8, 27 and 21 for he plus series: (a) The nework diagram (b) The nework simulaion and gene expression predicion over fuure ime. Solid markers represen observaions. (b) predicions (a) Fig. 2. The idenified bes GRN of gene 33 (elomerase) and genes 8, 27 and 21 for he minus series: (a) The nework diagram (b) The nework simulaion and gene expression predicion over fuure ime. Solid markers represen observaions. (b)

9 Gene Regulaory Nework Discovery from Time-Series Gene Expression Daa A Compuaional Inelligence Approach Building a global GRN of he whole gene se ou of he GRNs of smaller number of genes (Puing he pieces of he puzzle ogeher) Afer many GRNs of smaller number of genes are discovered, each involving differen genes (wih a differen frequency of occurring), hese GRNs can be pu ogeher o creae a GRN of he whole gene se. Represenaion and illusraion for he op five (fies) GRNs from our experimen are shown in Fig3 and Table 3 respecively. Fig. 3. The five highes likelihood GRN models found by GA in he plus series are pu ogeher. Table 2. Illusraion of op five fies GRNs (plus series) GRN Number GRN idenified 1 ( ) 2 ( ) 3 ( ) 4 ( ) 5 ( ) 4 Conclusions In his work, we propose a novel mehod ha inegraes Kalman Filer and Geneic Algorihm for he discovery of GRN from gene expression observaions of several ime series (in his case hey are wo) of small number of observaions. As a case sudy we have applied he mehod for he discovery of GRN of genes ha regulae elomerase in wo sub-clones of he human leukemic cell line U937. The ime-series conain 12,625 genes, each of which sampled 4 imes a irregular ime inervals, bu only 32 genes of ineres are deal wih in he paper. The mehod is designed o deal effecively wih irregular and scarce daa colleced from a large number of variables (genes). GRNs are modelled as discree-ime approximaions of firs-order differenial equaions and Kalman Filer is applied o esimae he rue gene rajecories from he irregular observaions and o evaluae he likelihood of he GRN models. GA is applied o search for smaller subse of genes ha are probable in forming GRN using

10 10 Nikola K. Kasabov1, Zeke S. H. Chan1, Vishal Jain1, Igor Sidorov2 and Dimier S. Dimirov2 he model likelihood as an opimizaion objecive. The biological implicaions of he idenified neworks are complex and currenly under invesigaion. References 1. Baeck, T., D. B. Fogel, e al.: Evoluionary Compuaion I and II. Advanced algorihm and operaors. Brisol, Insiue of Physics Pub (2000) 2. Bay, J. S. (ed.): Fundamenals of Linear Sae Space Sysems, WCB/McGraw-Hill (1999) 3. Bolouri, J. M. B. a. H. (eds.): Compuaional modelling of Geneic and Biochemical Neworks. London, The MIT Press (2001) 4. Brown, R. G. (ed.): Inroducion o Random Signal Analysis and Kalman Filering, John Wiley & Son (1983) 5. Brownsein, M. J., Tren, J.M., and Boguski, M.S., Funcional genomics In M. Paerson and M. Handel (eds.): Trends Guide o Bioinformaics (1998) Collado-Vides, J.: A ransformaional-grammar approach o sudy he regulaion of gene expression, J. Theor. Biol. 136 (1989) Dorf, R. and R. H. Bishop: Modern Conrol Sysems, Prenice Hall (1998) 8. Fields, S., Kohara, Y. and Lockhar, D. J.: Funcional genomics. Proc Nal. Acad. Sci USA 96 (1999) Friedman, L., Nachman, Pe'er: Using Bayesian neworks o analyze expression daa. Journal of Compuaional Biology 7 (2000) Goldberg, D. E. (1989). Geneic Algorihms in Search, Opimizaion and machine Learning Reading, MA, Addison-Wesley 11. Hofesad, R. a. M., F.: Ineracive modelling and simulaion of biochemical neworks Compu. Biol Med. 25 (1995) Holland. H.: Adapaion in naural and arificial sysems, The Universiy of Michigan Press, Ann Arbor, MI (1975) 13. Kasabov, N. and D. Dimirov: A mehod for gene regulaory nework modelling wih he use of evolving connecionis sysems. ICONIP - Inernaional Conference on Neuro- Informaion Processing, Singapore, IEEE Press (2002) 14. Likhoshvai, V. A., Maushkin, Yu G., Vaolin, Yu N. and Bazan, S. I: A generalized chemical kineic mehod for simulaing complex biological sysems. A compuer model of lambda phage onogenesis." compuaional echnol. 5, issue 2 (2000) Loomis, W. F., and Sernberg, P.W.: Geneic neworks. Science 269 (1995) Mc Adams, H. H. a. A. A.: Sochasic mechanism in gene expression. Proc. Nal. Acad. Sci. USA 94 (1997) Muhlenbein, H.: How geneic algorihms really work: I. muaion and hillclimbing. Parallel Problem Solving from Naure 2. B. Manderick. Amserdam, Elsevier (1992) 18. Sanchez, L., van Helden, J. and hieffry, D.: Esablishmen of he dorso-venral paern during embryonic developmen of Drosophila melanogaser. A logical analysis. J. Theor. Biol. 189 (1997) Tchuraev, R. N.: A new mehod for he analysis of he dynamics of he molecular geneic conrol sysems. I. Descripion of he mehod of generalized hreshold models. J. Theor. Biol. 151 (1991) Thieffry, D.: From global expression daa o gene neworks. BioEssays 21 issue 11 (1999) Thieffry, D. a. T. R.: Dynamical behaviour of biological regulaory neworks-ii. Immuniy conrol in baceriophage lambda. Bull. Mah. Biol 57 (1995)