Semilinear high-dimensional model for normalization of microarray data: a theoretical analysis and partial consistency

Transcription

1 Semilinear high-dimensinal mdel fr nrmalizatin f micrarray data: a theretical analysis and partial cnsistency Jianqing Fan, Heng Peng and Ta Huang September 3, 2004 Abstract Nrmalizatin f micrarray data is essential fr remving experimental biases and revealing meaningful bilgical results Mtivated by a prblem f nrmalizing micrarray data, a Semilinear n-slide Mdel (SLM) has been prpsed in Fan et al(2004) T aggregate infrmatin frm ther arrays, SLM is generalized t accunt fr acrss-array infrmatin, resulting in an even mre dynamic semiparametric regressin mdel This mdel can be used t nrmalize micrarray data even when there is n replicatin within an array We demnstrate that this semiparametric mdel has a number f interesting features: the parametric cmpnent and the nnparametric cmpnent that are f primary interest can be cnsistently estimated, the frmer pssessing parametric rate and the latter having nnparametric rate, while the nuisance parameters can nt be cnsistently estimated This is an interesting extensin f the partial cnsistent phenmena bserved by Neyman and Sctt (948), which itself is f theretical interest The asympttic nrmality fr the parametric cmpnent and the rate f cnvergence fr the nnparametric cmpnent are established The results are augmented by simulatin studies and illustrated by an applicatin t the cdna micrarray analysis f the neurblastma cell in respnse t the macrphage migratin inhibitry factr (MF) Jianqing Fan is Prfessr, and Heng Peng is pstdctral fellw, Department f Operatin Research and Financial Engineering, Princetn University, Princetn, NJ Ta Huang is pstdctral assciate, Department f Epidemilgy and Public Health, Yale University Schl f Medicine, New Haven, CT 065 The paper was partially supprted by the NSF grant DMS and NH grant R0-HL69720 The authrs wuld like t thank the editr, assciate editr and three annymus referees fr helpful cmments that led t significant imprvement f the presentatin f the paper

2 ntrductin DNA micrarrays mnitr the expressin f tens f thusands f genes in a single hybridizatin experiment using lignucletide r cdna prbes The technique has been widely used in many bimedical research and bilgical studies A challenge in analyzing micrarray data is the systematic biases due t the variatins in experimental cnditins such as the efficiency f dye incrpratin, intensity effect, the cncentratin f DNA n arrays, the amunt f mrna, variability in reverse transcriptin, batch variatin, amng thers Nrmalizatin is required in rder t remve the systematic effects f cnfunding factrs s that meaningful bilgical results can be btained There are several useful nrmalizatin techniques that aim at remving the systematic biases such as the dye, intensity, print-tip blck effects The simplest methd is the glbal nrmalizatin methd featured in sftware package such as GenePix40 and analyzed by Krll and Wölfl (2002) Such a technique implicitly assumes that there is n print-tip blck effect and n intensity effect Withut such an assumptin, the methd is statistically biased The lwess methd in Dudit et al(2002) significantly relaxes the abve assumptin But, it assumes that the average expressin levels f up- and dwn-regulated genes at each intensity level are abut the same in each print-tip blck This assumptin is relaxed further by Tseng et al(200) t nly a subset f mre cnservative genes based n a rank invariant selectin methd As admitted in Tseng et al(200), the methd is nt expected t be useful when there are far mre genes that are up-regulated (r dwn-regulated) Such situatins can ccur when cells are treated with sme reagents (Grssleau et al2002, Fan et al2004) n an attempt t relax further the abve bilgical assumptin, Huang et al(2003) and Huang and Zhang (2003) intrduce a semi-linear mdel t accunt fr the intensity effect and t aggregate infrmatin frm ther arrays t assess the intensity effect The methd is expected t wrk well when the gene effect is the same acrss arrays n an attempt t relax further the afrementined statistical and bilgical assumptins in the cdna micrarray nrmalizatin, Fan et al(2004) develpe a new methd t estimate the intensity and print-tip blck effects by aggregating infrmatin frm the replicatins within a cdna array cdna micrarray chips are usually cnstructed by dipping a printer head, which cntains 6 sptting pins, int a 96-well plate cntaining cdna slutins; printing these 6 spts n the slide; washing the sptting pins; dipping them int different 6 wells and printing again, and s n See Craig et al(2003) fr details Fr the specific designs f cdna micrarrays used in Fan et al(2004), there are clnes that are printed twice n the cdna chips The lcatins f these 222 replicatins appear randm in the 32 blcks (see 42) n ther wrds, replicatins are nt achieved by printing twice the same 6 spts n a printer head, but by the cnstructin f the wells in plates themselves The replicatins f the clnes in the cdna chips cntain a lt f infrmatin abut the systematic biases such as the print-tip blck and intensity effects n fact, fr tw identical clnes f cdna in the same slide, apart frm the randm errrs, the expressin ratis shuld be the same bserved differences f expressin ratis tell us a lt f infrmatin abut the print-tip blck and intensity effects The seemly-randm patterns f replicatins enable ne t unveil the 2

3 print-tip blck effect This can nt be achieved if nly the same 6 spts in a well plate are printed twice T put the abve prblem in a statistical framewrk, let G be the number f genes, and be the number f replicatins f gene g within an array ( shuld depend n g, as majrity f genes d nt have replicatins) Fllwing Dudit et al(2002), let R gi and G gi be the red (Cy5) and green (Cy3) intensities f the gth gene in the ith replicatin, respectively Let Y gi be the lg-intensity rati f red ver green channels f the gth gene in the ith replicatin, and let X gi be the crrespnding average f the lg-intensities f the red and green channels That is, Y gi = lg 2 R gi G gi, X gi = 2 lg 2(R gi G gi ) T mdel the intensity and print-tip blck effects, we cnsider the fllwing high-dimensinal partial linear mdel fr micrarray data Y gi = α g + β rgi + γ cgi + m(x gi ) + ε gi, () where α g is the treatment effect assciated with the gth gene, r gi and c gi are the rw and clumn f print-tip blck where the gth gene f the ith replicatin resides, β and γ are the rw and clumn effects with cnstraints r β i = 0 and c γ j = 0, i= j= where r and c are the number f rws and clumns f the print-tip blck, m( ) with cnstraint Em(X gi ) = 0 is a smthing functin f X representing the intensity effect, and ε gi is randm errr with mean zer and variance σ 2 (X gi ) n ur illustrative example in Sectin 42, there are 9,968 genes in an array, residing in 8 4 blcks with r = 8 and c = 4 Amng thse, there are genes with tw replicatins Fr thse genes withut replicatins, since α g is free, they prvide n infrmatin abut the parameters β and γ and the smth functin m( ) We need t estimate the parameters frm the genes with replicatins With slight abuse f ntatin, fr this illustrative example, G = and = 2 Fr the nrmalizatin purpse, ur aim is t find a gd estimate f the print-tip blck and intensity effects Let ˆβ, ˆγ and m( ) be gd estimates fr mdel () Then, the nrmalizatin is t cmpute Y g = Y g β rg γ cg m(x g ) (2) fr all genes nterplatins and extraplatins are needed t expedite the cmputatin when m( ) is estimated ver a set f fine grid pints Accrding t mdel (), Yg α g + ε g, in which the effects f cnfunding factrs have been remved Thus, as far as the prcess f the nrmalizatin is cncerned, the parameters β and γ and the functin m( ) are f primary interest and the parameters {α g } are nuisance parameters Of curse, in the analysis f treatment effect n genes, the parameters {α g } represent bilgical fld changes and are f primary interest Mdel () has a much wider spectrum f applicatins than it appears First f all, if there is n replicatin within an array but there are 4 (say) replicatins acrss arrays, by imaging a super 3

4 array that cntains these 4 arrays, within-array replicatins are artificially created n this case, is the number f arrays and G is the number f genes per array The basic assumptin behind this methd is that the treatment effect n the genes remains the same acrss arrays This is nt an unreasnable assumptin when the same experiment is repeated several times Secndly, by remving the rw and clumn effects and applying mdel () directly t each blck f micrarrays, resulting in Y gib = α g + m b (X gi ) + ε gib, (3) the mdel allws nn-additive effect between the intensity and blcks [The index b can be remved frm mdel (3) and hence this mdel becmes a submdel f ()] n this case, G is the number f genes within a blck Fr example, if there are 624 genes in a blck and there are 4 replicatins f arrays, then G = 624 and = 4 Thirdly, the idea can als be adapted t nrmalize the Affymetrix arrays by imaging treatment and cntrl arrays as the utputs frm green channels and red channels This will enable us t remve intensity effects in the Affymetrix arrays Finally, by thinking rws as blcks and deleting the clumn effects, mdel () can accmmdate nnadditive clumn and rw effects The additivity in mdel () is t facilitate the applicatins in which G is relatively small The challenge f ur prblem is that the number f nuisance parameters is large n fact, fr many practical situatins, = 2 and G can be large, in an rder f hundreds r larger S ur asympttic results are based n the assumptin that G This is in cntrast with the assumptin in Huang and Zhang (2003) where tends t infinity The number f nuisance parameters in () grws with the sample size n ur illustrative example, half f the parameters are nuisance nes The questin is if the parameters f primary interest can be cnsistently estimated in the presence f a large number f nuisance parameters and hw much they cst us t estimate these parameters Such a kind f prblem is prly understd and a thrugh investigatin is needed T prvide mre insights abut the prblem, cnsider writing mdel () in the matrix frm as Y n = B n α n + Z n β + M + ɛ n, n = G, (4) where Y n = (Y,, Y n ) T, B n is an n G design matrix, Z n is an n d randm matrix with d being the sum f the numbers f rw and clumn, β = (β,, β r, γ,, γ c ) is the print-tip blck effect, M = (m(x ),, m(x n )) T is the intensity effect and ɛ n = (ε,, ε n ) T The thery n the partial linear mdel is usually based n the assumptin that G is fixed r at least G/n tends t zer at certain rate (see Härdle et al, 2000) Hwever, in ur applicatin, α n can nt be cnsistently estimated as G/n = / in (4) t is nt clear whether the parameters β and the functin m( ) can be cnsistently estimated The answer might nt be affirmative fr general matrix B n Fr ur applicatin, the matrix B n is in a specific frm: B n = G, where is the Krnecker prduct, G is the G G identity matrix and is a vectr f length with all elements Using such a structure, the answer is affirmative n the next sectin, we will shw that β can be estimated at the parametric rate n /2 and m( ) can be estimated at the nnparametric rate n 2/5 Further, 4

5 we derive the asympttic nrmality f β and describe exactly the cst fr estimating the nuisance parameters Our results are interesting extensins f the partial cnsistent phenmenn studied by Neyman and Sctt (948) They shw that when the number f nuisance parameters grw with sample size, ne parametric cmpnent can be cnsistently estimated, while the ther part can nt Our results are f theretical interest in their wn right, in additin t prviding insights and methdlgical advance fr the prblem f micrarray nrmalizatin The remainder f the article is rganized as fllws n Sectin 2 we derive the prfile leastsquares estimatrs fr the parameters α n, β and the functin m( ) We further demnstrate that the prpsed estimatrs fr β and m( ) are cnsistent and admit ptimal rates f cnvergence Sectin 3 extends mdel () t aggregate infrmatin frm ther arrays Bth simulated and real data examples are given in Sectin 4 Technical prfs and regularity cnditins are relegated t the Appendix 2 Prfile least-squares and asympttic prperties 2 Prfile least-squares estimatr n this sectin we derive the prfile least-squares estimatrs fr α n, β and the functin m( ) Fr any given α n and β, (4) can be written as Y n B n α n Z n β = M + ɛ n (2) Thus, adpting lcal linear regressin technique, the estimatr f M is M = S(Y n B n α n Z n β), where S is a smthing matrix and depends nly n the bservatins {X i, i =,, n} explicit frm f S is shwn in the Appendix fr the lcal linear smther (Fan, 992) Substituting M int (2), we have Ỹ n = B n α n + Z n β + ɛ n, where Ỹ n = ( S)Y n, Bn = ( S)B n and Z n = ( S)Z n This is a synthetic (but nt a true) linear mdel By the least-squares methd and a slight cmplex cmputatin (Ra and Tutenburg 999), we have estimates fr β and α n as fllws: The β = ( Z T n Z n Z T n P Bn Zn ) ZT n ( P Bn )Ỹ n, (22) and α n = ( B T n B n ) BT n (Ỹ n Z n β), (23) where P Bn = B n ( B T n B n ) BT n is a prjectin matrix f B n The abve prfile least-squares estimatrs can be cmputed by the fllwing iterative algrithm: 5

6 Step : Given β and m( ) (assume their initial values t be zers), estimate α g by α g = (Y gi β rgi γ cgi m(x gi )) i= Step 2: Given α n and m( ), estimate β by fitting the fllwing linear mdel Y gi α g m(x gi ) = β rgi + γ cgi + ε gi Center the estimated cefficients f β and γ s that their averages are zer Step 3: Given α n and β, estimate the functin m( ) by smthing Y gi α g β rgi γ cgi n X gi Center the estimated functin t have mean value zer Step 4: Cntinue Steps -3 until cnvergence n ur implementatin, it takes nly a cuple f iteratins fr the algrithm t cnverge The advantage f this estimatin algrithm is that it effectively separates the estimatin prblem int tw parts, s that the nnparametric cmpnent can be estimated lcally, while the parametric cmpnents can be estimated glbally by using all the data The methd is in well cntrast with the spline methd f Huang et al (2003) First f all, the parameters α n and β are estimated ne by ne This avids inverting any large matrix, and is very useful fr the high dimensinal micrarray data Secndly, α n and β can be estimated by the weighted least-squares in presence f heterscedasticity [eg, the nise level depends smthly n an unknwn functin σ(x gi )] Thirdly, the smthing parameters in Step 3 can be selected by using an existing technique such as the pre-asympttic substitutin methd f Fan and Gijbels (995) and the empirical-bias methd f Ruppert (997) amng thers n ur implementatin, the empirical-bias methd f Ruppert (997) is emplyed 22 Asympttic prperties n parametric cmpnent T facilitate the presentatin and technical prfs, we assume that {(r gi, c gi, X gi, ε gi ), i =,,, g =,, G} are a randm sample frm a ppulatin Mre generally, we assume that the randm variables {(Z j, X j, ε j ) : j =,, n} in (4) are a randm sample frm a ppulatin Fr mdel (), the randm variable Z j is the indicatr variable assciated with the clumn and rw effects These assumptins are made t facilitate theretical derivatins Hwever, the scpe f applicatins ges beynd these technical assumptins, as demnstrated in ur simulatin studies Fr simplicity, assume fr the mment that the mdel (4) is hmscedastic, namely, var(ε j Z j, X j ) = σ 2 Then we have the fllwing asympttic prperty fr β 6

7 Therem Under the regularity cnditins in the Appendix, the prfile least-squares estimatr f β is asympttically nrmal, ie n( β β) with Σ = E{Z E(Z X)} T {Z E(Z X)} D N (0, σ2 Σ ) When α n is knwn, mdel (4) reduces t the partial linear mdel n this case, the asympttic variance is Σ (see, fr example, Speckman 988 and Carrl et al997) which is the semiparametric infrmatin bund Since there are G nuisance parameters, they cst us at least G data pints t estimate them and the remaining degree f freedm is n G = n( )/ The factr /( ) is the price that we have t pay fr estimating the nuisance parameters α n This price factr is the minimum that we can have T appreciate this, let us cnsider a very simple mdel Y gi = α g + βu gi + ε gi, where ε gi N(0, σ 2 ), U gi is distributed with EU gi = 0 and EUgi 2 = The efficient infrmatin bund fr estimating β is n σ 2 /( ), a factr /( ) larger than the case where α g s are knwn Our theretical result is derived under the randm design f Z, it als hlds fr fixed design f Z satisfying certain mathematical cnditins This is indeed shwn via simulatin studies in Examples 3 and 4 in 4 We nw cnsider the situatin f heterscedastic errr in the mdel (4), ie ɛ n = (σ(x )ε,, σ(x n )ε n ) T with a cntinuus standard deviatin functin σ( ) n the supprt f X Suppse that we ignre the heterscedasticity in the estimatin prcedure We have the fllwing therem fr the asympttic prperty f β Therem 2 Under the regularity cnditins in the Appendix, the prfile least-squares estimatr f β is asympttically nrmal, ie where with Σ given in Therem, V = n( β β) D 2 N (0, ( ) 2 Σ V Σ ), ( )2 2 Eσ 2 (X){Z E(Z X)} T {Z E(Z X)} + 2 Eσ 2 (X) Σ Therem 2 examines the impact f heterscedasticity n the rdinary least-squares estimate The heterscedasticity is nt explicitly taken int accunt fr the fllwing reasns First f all, even if the cnditinal variance functin is knwn, after standardizatin, ur mdel structure will be changed the special structure f B n in (3) des nt hld any mre n ur simulatin studies in Example 2, there is nt much imprvement by using the weighted least-squares n fact, it is nt clear t us whether the weighted least-squares must utperfrm the rdinary least-squares fr this special class f mdels Further research is needed 7

8 23 Nnparametric part The purpse f this paper is t shw that the nnparametric functin m( ) and parametric parameter β can be estimated cnsistently and efficiently Therems and 2 have already shwn that β can be estimated at rt-n rate, which is negligible fr nnparametric estimatin T simplify technical derivatins withut lsing the essential ingredient, we assume that β is knwn Therefre, mdel (4) can be simplified as Y n = B n α n + M + ɛ n, (24) where B n = G and n = G nstead putting identifiability n the functin m, we impse the identifiability cnditin G i= α i = 0 t facilitate the technical arguments The prfile least-squares estimatr can be regarded as the iterative slutin t the fllwing equatin (see the algrithm in 2) [ P where S ] [ Bn α n M ] = [ P S ] Y n, P = B n ( T /n)(b T n B n ) B T n = ( T /n)b n (B T n B n ) B T n Accrding t the results f Opsmer and Ruppert (997), the estimate f M has the fllwing explicit frm: M = { ( SP) ( S)}Y n, (25) prvided the inverse matrix exists Then we have the fllwing asympttic prperty fr M Therem 3 Under the regularity cnditins in the Appendix, the risk f the prfile least-squares M is bunded as fllws: MSE{ ˆm(x) X,, X n } where = n( ) 2 i= ( µ 2 2 h 4 ( ) 2 MSE{ ˆm(x) X,, X n } = n and Ω is the supprt f the randm variable X { µ 2 2 h 4 4 {m (X i )} 2 + σ2 ν 0 nhf(x i ) 4 E{m (X)} 2 + σ2 ν 0 Ω nh } ( + p h 4 + nh ) + p ( h 4 + nh E{[ ˆm(X i ) m(x i )] 2 X,, X n } i= Fr the situatin f heterscedastic errrs, if we ignre the errr structure and apply the rdinary least-squares, then we have the fllwing therem: Therem 4 Under the regularity cnditins in the Appendix, the full-iterative backfitting estimatr M has ), ) + ( µ 2 MSE{ ˆm(x) X,, X n } ( 2 ) 2 4 E{m (X)} 2 h 4 { ( )Eσ 2 (X) Ω ( } ) σ 2 ν0 ) ( (x)dx + p h 4 + ) nh nh Ω 8

9 t is clear frm the abve tw therems, the nnparametric cmpnents achieve the ptimal rate f cnvergence O(n 2/5 ) when the bandwidth h is f rder n /5 3 Aggregatin acrss arrays n the last sectin, the intensity effect and the gene effect are estimated by using the infrmatin within ne slide An advantage f this is that the arrays are allwed t have different gene effect, namely, α g can be slide-dependent This ccurs when samples were drawn frm different subjects n many ther situatins, the samples may cme frm the same subject n this case, it is natural t assume that the treatment effects n genes are the same acrss arrays and ne can aggregate the infrmatin frm ther arrays This kind f aggregatin idea has appeared in wrk f Huang and Zhang (2003) fr a different semiparametric mdel Mdel (3) is an aggregated mdel, allwing interactins between blcks and intensities Drpping the blck label b, it becmes Y gj = α g + m(x gj ) + ε gj, j =,, J, (3) where J is the number f arrays This mdel is the same as (24) and Therems 3 and 4 give asympttic perfrmance fr the intensity effect (at each blck) f m n mdel (3), the intensity effects are the same acrss arrays The results can be generalized t the array-dependent mdel Y gj = α g + m j (X gj ) + ε gj, j =,, J (32) The functins m j can be estimated at rate O(n 2/5 ) A generalizatin f mdel () is the fllwing semiparametric mdel Y gij = α g + β j,rgi + γ j,cgi + m j (X gij ) + ε gij, (33) g =,, G, i =,, and j =,, J with J being the number f arrays, where r gi and c gi are respectively the rw and clumn f the print-tip blck where the gth gene f the ith replicatin resides (this usually des nt depend n the array), β j = (β j,,, β j,r, γ j,,, γ j,c ) and m j ( ) represent the blck effect and intensity effect fr each array j The mdel (33) is nt identifiable when there is n replicatin within an array ( = ) and is identifiable when there is a replicatin > Since all arrays share the same amunt f gene effect α g, the nuisance parameters α g can be estimated mre accurately The questin is hw much better the parameters f interest β j and m j ( ) can be estimated by using the aggregatin The algrithm in 2 can be mdified as fllws: Step : Given β j and m j ( ) (assume their initial values t be zers), estimate α g by averaging ver with respect t i and j Y gij = Y gij β j,rgi γ j,cgi m j (X gij ) 9

10 Step 2: Given α n and m j ( ), estimate β j using the fllwing linear mdel Y gij = Y gij α g m(x gij ) = β j,rgi + γ j,cgi + ε gij This is dne fr each separate j Center estimated β s and γ s t have zer mean Step 3: Given α n and β j, estimate functin m j ( ) with Y gij = Y gij β j,rgi γ j,cgi = m j (X gij ) + ε gij Again, this is dne fr each separate j Center ˆm j s that its average is zer Step 4: Cntinue Steps -3 until cnvergence Assume that the data frm each array are iid sample with hmscedastic errr Then, the prblem is similar t (), but the cst fr estimating nuisance parameters is shared by J arrays One key difference is that the rws r gi and clumns c gi d nt depend n the array j This makes the prblem under the study harder Put mdel (33) int the matrix frm as Y = Bα + Z β + M + ɛ where Y = (Y T,, Y T J )T, Y j = (Y j, Y 2j,, Y Gj ) T, B = (B,, B J ) T, B j = T G T, Z = diag(z,, Z J ), β = (β T,, β T J )T, M = (M T,, M T J )T, M j = {m j (X gij ), g =, 2,, G, i =,, } T, with Z = Z 2 = = Z J = Z as the gene psitin in each array remains the same Assume that the cnditinal distributin f Z given (X i, X j ) are the same fr all i j Fllwing the prf similar t that f Therem, we have the fllwing asympttic nrmality fr the aggregated estimatr The asympttic is based n the assumptin that G Therem 5 Under the regularity cnditins in the Appendix, the prfile least-squares estimatr f β j is asympttically nrmal, ie G( βj β j ) D N (0, σ 2 Σ j ), where Σ = E{Z E(Z X )} T {Z E(Z X 2 )}, Σ = E{Z E(Z X)} T {Z E(Z X)} and Σ j = ( J J Σ + J Σ ) ( + Σ (J J J Σ + ) ( J J Σ ) J Σ + J Σ ) with X, X, and X 2 having the same distributin as that X gij Despite its cmplicated asympttic expressin, Therem 5 shws the extent t which the cst f estimating nuisance parameter α g can be reduced T simplify ur results, we cnsider the face that Z gij is independent f X gij Then, Σ = Σ and Σ j = J( ) + Σ (34) J( ) 0

11 t shws that mdel is nt identifiable when = Since the sample size fr estimating β j is G when α g s are knwn, the cst fr estimating α g s fr each array is abut G GJ( ) J( ) + = G J( ) + data pints Fr example, if = 2, J = 6 and G = as in the illustrative example in 42, then the lst f degree f freedm due t estimating nuisance parameters reduces frm t 37 Hwever, the efficiency f the intensity effect m j can nt be imprved very much as it is estimated separately frm each slide ndeed, frm the asympttic pint f view, β can be treated as if they were knwn fr estimating m j ( ), whether r nt the infrmatin frm ther arrays are aggregated Since the errrs in estimating m j ( ) dminate thse in estimating the blck effects, the accuracy fr nrmalizatin (2) can nt be imprved very much by aggregating infrmatin frm ther arrays This gives theretical supprt n the array-wise nrmalizatin methd f Fan et al(2004) Their methd has additinal advantage f cmputatinal expedience and rbust t the assumptin that the gene effects remain the same acrss arrays Aggregatin des give us ne imprtant advantage: t reduces the cst fr estimatin nuisance parameters per array With aggregated sample size JG, which amunts t 332 fr ur illustrative example, we may be willing t relax the additive clumn and rw effect Namely, we may extend mdel (33) t the mre flexible mdel Y gij = α g + δ j,bgi + m j (X gij ) + ε gij, (35) where {b gi } (ranging frm t 32 in ur applicatin) is the blck where the gene g with repetitin i resides and {δ j,k } measures the blck effect f the jth array, cnsisting f J(cr ) parameters This can als be regarded as a specific mathematical mdel f (33) by thinking β = δ and γ = 0 Thus, Therem 5 cntinues t apply and their estimatin errrs are negligible in cmparisn with thse in estimatin m j ( ) Mdel (33) is a specific case f (35) with additive rw and clumn effects Thus, mdel (35) reduces pssible mdeling biases The nrmalizatin (2) nw becmes fr each slide Y g = Y g ˆδ bg ˆm(X g ), (36) The abve results are based n the assumptin that Z = = Z J fr micrarray applicatins On the ther hand, it is pssible t design the case that Z, Z 2,, Z J are independent This amunts t using mdel (33) with c rgj and r rgj The fllwing therem gives the result n this specific case Therem 6 Under the regularity cnditins in the Appendix, the prfile least-squares estimatr f β j is asympttically nrmal, ie G(J )/J( βj β j ) where Σ j = E{Z j E(Z j X j )} T {Z j E(Z j X j )} D N (0, σ 2 Σ j ),

12 This specific mdel is f theretical interest, with pssible applicatins t ther statistical prblems Fr this specific mdel, Σ = 0 in Therem 5, and Therem 6 can be deduced frm Therem 5 With this specific design, the cst fr estimating α g s fr each array reduces further t G/J data pints Fr example, if = 2, J = 6 and G =, then the lst f degree f freedm due t estimating nuisance parameters reduces frm t 85 This cmpares with 37 data pint with fixed design Z = = Z J mentined abve 4 Simulatin and applicatin n this sectin, we use several simulated examples t augment the partial cnsistent phenmenn demnstrated in the last tw sectins We then cnclude this sectin by an applicatin f the prpsed methd t the nrmalizatin f the micrarray data arising frm the study f the neurblastma cell in respnse t the stimulatin f the macrphage migratin inhibitry factr (MF), a grwth factr 4 Simulatins Our theretical results are illustrated empirically by fur examples The first tw examples study the situatin under which the genes with replicatins are randmly placed n arrays One is hmscedastic mdel and the ther is heterscedastic mdel The last tw examples shw that ur results cntinue t apply t fixed designs As ur mdels and the partial cnsistent results are mtivated by the analysis f micrarray data, the validity f the randmness assumptin arises fr replicated genes We use Example 3 t demnstrate that ur methds cntinue t wrk fr the replicatins similar t ur illustrative example in 42 Finally, in Example 4, ur methd is applied t the case in which n gene has replicatins within an array and the design f genes is fixed n all examples, we assume that there are 32 print-tip blcks with 4 rws and 8 clumns The perfrmances f α n, β and m( ) are assessed by the mean squared errrs (MSE): MSE( α n ) = G MSE( m) = G G ( α g α g ) 2, g= MSE( β) = r + c β β 2, G g= i= { m(x gi ) m(x gi )} 2 The mean squared errrs are examined by varying G and T examine the impact f the heterscedasticity n the efficiency f parameters, we als cnsider the weighted least-squares methd, in which steps and 2 are implemented by using the weighted least-squares with the cnditinal variance functin estimated by smthing the squared residuals n X gi (see Fan and Ya, 998) Example n this example we chse G = 00, 200, 400, 800 and = 2, 3, 4 Fr each pair f (G, ), we simulate N = 200 data sets frm mdel () T examine hw much the aggregated 2

13 methd in 3 can imprve the estimatin f the intensity effect m( ) and print-tip blck effect (β and γ), we simulate data frm mdel (33) with the same print-tip blck effect and intensity effect as in mdel () We assume that there are J = 4 arrays available fr us t aggregate the infrmatin We repeat the simulatin N = 50 times, each time cnsisting f J = 4 arrays fr aggregatin The details f simulatin scheme fr this example are as fllws: α n : The expressin levels f the genes are generated frm the standard duble expnential distributin β: Fr the rw effects, first generate {β i, i =,, 4} frm N(0, 5), and then set β i = β i β, which will guarantee that 4 i= β i = 0 The clumn effects are generated in the same way X: The intensity is generated frm a mixture distributin We generate x frm prbability distributin 0004(x 6) 3 (6 < x < 6) with prbability 7 and frm unifrm distributin ver [6, 6] with prbability 3 m( ): Set the functin m(x) = 5(sin X 2854), whse expectatin is zer Z: Fr each given gene, its assciated blck is assigned at randm at ne f 32 print-tip blcks ε: ε gi is generated frm the standard nrmal distributin Table : MSEs f Example, OLS & WLS Estimatins, N=200 Ordinary Least-Squares Weighted Least-squares G=00 G=200 G=400 G=800 G=00 G=200 G=400 G= m β α This is a hmscedastic mdel The estimatin prcedure that is described in Sectin 2 is used t estimate α n, β and m( ) Table presents the MSEs f the rdinary least-squares and the weighted least-squares estimatrs fr α n, β and m( ), respectively Frm Table, it can be seen that when the number f replicatins is fixed, the MSEs f m( ) and β decrease as the number f genes increases, which indicates the cnsistency f the estimatrs f m( ) and β Hwever, the MSEs f α n are very stable as the number f genes increases, which demnstrates the incnsistency 3

14 Table 2: MSEs f Example, Aggregatin Acrss Arrays Estimatin, N=50, J=4 G=00 G=200 G=400 G= m β α f the estimatr fr α n Furthermre, t visualize the MSEs and the cnvergence rates, these MSEs are als depicted in Figure, where lg(mse) lg(/( )) is pltted against lg(n) Nte that n = G and the factr /( ) is used t crrect the intercept term (See Therem ) The figure shws that β pssesses the parametric rate n /2 and m( ) has the nnparametric rate n 2/5 Table 2 presents the MSEs fr α n, β and m( ), which are als pltted in Figure, when the infrmatin frm 4 arrays is aggregated Clearly, α n is estimated mre accurately by aggregating infrmatin acrss arrays with the assumptin that the gene effects remain the same acrss arrays n cntrast, the imprvements n the estimatin f β and m( ) are relatively smaller Accrding t Therems and 5, the asympttic relative efficiency fr estimating β is (4 4)/(4 ) The relative efficiencies are in line with thse shwn in Tables and 2 T shw the effectiveness f estimated functins and parameters, Figure als shws tw randmly selected estimates m( ) fr G = 400 and = 2 Figure 2 summarizes the bxplts f rw { ˆβ k } and clumn {ˆγ k } effects based n 200 simulatins with G = 400 and = 2 The biases f these estimates are clearly negligible and cefficients are estimated with similar accuracy This is cnsistent with ur design f simulatins Fr simplicity, we nly present the results based n the rdinary least-squares methd Example 2 n this example we als chse G = 00, 200, 400, 800 and = 2, 3, 4 Fr each pair f (G, ), we simulate N = 200 data sets frm the mdel () T examine the effectiveness f using infrmatin acrss arrays, we simulate N = 50 data sets frm the mdel (33), each cnsisting f J = 4 arrays with parameters taken frm mdel () The parameters in this example are taken t mimic the real data in the next sectin The details f simulatin scheme is: α n : The expressin levels f the first 50 genes fllw standard duble expnential, and the rest are zers 4

15 lg(mse) lg(/( )) (a) m( ) lg(mse) lg(/( )) (b) β Lg(n) Lg(n) 0 05 (c) α OLS WLS ACROSS 2 (d) Estimated m( ) lg(mse) Lg(n) Figure : (a)-(c) Plts f MSEs f m( ), β and α fr Example Ordinary least-squares(star); weighted least-squares (plus); aggregated methd (circle) The dtted lines are the regressin lines fr the MSEs f the three different estimatrs The slpes are shwn fr m( ) and β (d) The perfrmance f m( ) when G = 400 and = 2 The dt line is the true functin f m( ) and the slid lines are tw estimated functins 03 Bxplt Rw effects Clumn effects Figure 2: Bxplts f the estimated rw { ˆβ k } and clumn {ˆγ k } effects when G = 400 and = 2 fr Example 5

16 Table 3: MSEs f Example 2, OLS & WLS Estimatin, N=200 Ordinary Least-Squares Weighted Least-squares G=00 G=200 G=400 G=800 G=00 G=200 G=400 G= m β α Table 4: MSEs f Example 2, Aggregatin Acrss Arrays Estimatin, N=50, J=4 G=00 G=200 G=400 G= m β α β: The rw and clumn effects are fixed Set β r = (2, 5, 2, 5) and β c = (5, 25,, 075, 5, 25, 0, 075) X: Same as Example m( ): Set the functin m(x) = 0(2708 (6 X)/32), whse expectatin is zer Z: Same as Example ε: ε gi is generated frm the nrmal distributin with mean zer and variance σ 2 (X gi ) = (2 X gi ) 2 {X gi < 2} 6

17 This heterscedastic mdel cntains many features similar t thse in the real micrarray data Tables 3 and 4 shw similar results t thse f Tables and 2 They demnstrate that the estimatin f m( ) and β are cnsistent, while the estimatin f α n is nt Als nte that fr bth hmscedastic and heterscedastic mdels, the rdinary least-squares and weighted least-squares yield the similar results This is smewhat surprising But we speculate that the degree f heterscedasticity is nt big enugh s that the rdinary least-squares and the weighted least-squares perfrm analgusly Further, as pinted ut after Therem 2, it is nt clear if the weighted leastsquares must utperfrm the rdinary least-squares under the current mdel The results in Figure 3 are similar t thse f Figure Here, we nly present the results fr weighted least-squares estimatrs in Figure 3(d) ndubitably, the aggregatin imprves dramatically the estimate f α n t als imprves the accuracy f estimated intensity effect and print-tip blck effect lg(mse) lg(/( )) (a) m( ) lg(mse) lg(/( )) (b) β Lg(n) Lg(n) 2 25 (c) α OLS WLS ACROSS 3 2 (d) Estimated m( ) lg(mse) Lg(n) Figure 3: (a)-(c) Plts f MSEs f m( ), β and α Ordinary least-squares (star); weighted least-squares (plus); aggregated methd (circle) The dtted lines are the regressin lines fr the MSEs f the three different estimatrs The slpes are shwn fr m( ) and β (d) The perfrmance f m( ) when G = 400 and = 2 The dt line is the true functin f m( ) and the slid lines are tw estimated functins Example 3 n this example, we assess the impact f randmness assumptin t ur prpsed methd by fixing the replicated pairs thrughut simulatin We simulate N = 200 data sets frm mdel () fr pair (G =, = 2) The simulatin scheme is the same as Example 2 except the manner in which these 222 genes are placed nt micrarrays T be mre clse t reality, we mimic the real data in the next sectin and fix the print-tip blck psitins f these 222 genes thrughut simulatins The lcatins f the repeated pairs are identically the same as thse in the real data Table 5 cmpares the MSEs f α, β and m( ) under bth randm and fix designs 7

18 They are cmparable, which in turn indicates that the asympttic results cntinue t hld fr fixed designs (in which the randmness assumptin is vilated) Table 5: MSEs f Example 3, G=, =2, N=200 Ordinary Least-Squares Weighted Least-squares Fix Design Randm Design Fix Design Randm Design m β α Example 4 This example examines the effectiveness f nrmalizatin when there is n replicated genes within an array Data are simulated frm mdel (32) with α, X, m( ) and ε generated in the same manner as thse in Example 2 Mdel (32) are fitted fr different number f genes G within a blck f an array and fr different number f available arrays J fr aggregatin Table 6 presents the results t is clear when the number f available arrays increases, the MSEs fr m s (averaging MSE ver J arrays) decrease, since the average cst per array fr estimating α s decreases When G increases, the MSE s fr m decrease dramatically, which shws that m is a cnsistent estimate This is nt true fr the MSEs f α s, as α can nt be estimated cnsistently These are cnsistent with ur theretical results Table 6: MSEs f Example 4, N=00 G J=2 J=4 J=6 J=8 m α Applicatin The data set used here was cllected and analyzed by Fan et al(2004) The bilgical aim f the study is t understand hw genes are affected by the macrphage migratin inhibitry factr (MF) in neurblastma cells Neurblastma is the secnd cmmn paediatric slid cancer and accunts fr apprximately 5% cancer deaths MF plays a central rle in the cntrl f the hst inflammatry and immune respnse and is linked with fundamental prcesses that cntrl cell prliferatin, cell survival and tumr prgressin t is ver expressed in several human cancers T gain better understanding n the rle f MF in the develpment f neurblastma, the glbal gene expressin f neurblastma cell line stimulated with MF is cmpared with that f thse withut 8

19 Lcatins f printing-tip blcks Distributin f blcks (a) Distributin f rws (b) Distances between tw repetitins (c) (d) Figure 4: (a) Schematic representatin f the lcatins with replicatins; dt represents ne clne and triangle represents its replicatin (b) The distributin f the blcks where genes with repetitins reside (c) The pie chart fr the rws where genes with repetitins reside (d) The histgram fr the distances f the blcks fr tw repeated clnes MF stimulatin using cdna micrarrays The details f the design and experiments can be fund in Fan et al(2004) The cdna micrarrays used here cnsist f 9,968 clnes f sequence-validated human genes, printed n 8 4 print-tip blcks Amng them, cdna clnes f genes were printed twice n each array Figure 4(a) shws schematically the 32 print-tip blcks, with a pint in the blck indicating ne f these genes (dt represents ne clne and triangle represents its replicatin) These 222 clnes are nearly unifrmly distributed n the 32 blcks [see Figures 4(b) and 4(c)] Figure 4(d) shws the distributin, amng pairs f repeated genes, f the distance between tw repeated clnes Fr example, if ne gene is lcated in blck (3, 2) and the ther is lcated in blck (5, 3), then its distance is 4 + n the ntatin that we have intrduced, c = 8, r = 4, G = and = 2 Figures 5(a) and (b) shw the values f lg-ratis and lg-intensities fr the repeated genes fr a given array Since the clnes are identical, the differences in expressin prvide valuable infrmatin fr estimating the systematic biases The differences cme frm the lcatin f clnes in additin t the randm errrs Fllwing the wrk f Tseng et al(200), Dudit et al(2002), Huang et al(2003), we need 9

20 ndex f cdna Expressin Ratis Ratis fr repeated genes, H45-07 (a) ndex f cdna ntensity f Experssin ntensities f repeated genes (b) ntensity Rati ntensity versus rati (c) ntensity Rati ntensity vs rati, H45-07 (d) Quantiles f Standard Nrmal Ratis Distributin f ratis (e) Quantiles f Standard Nrmal Standardized ratis Distributin f Std ratis (f) Figure 5: (a) & (b) Observed lg-ratis and intensities fr pairs f repeated clnes (c) Fitted functin m alng with nrmalized (squares) and unnrmalized (triangles) ratis fr a given array are pltted against the lg-intensity (d) Nrmalized lg-ratis (vs their lg-intensities) with intensity and print-tip effect remved fr the given array [thick curves are the standard deviatin curves multiplied by 2 and dashed curve is the estimated functin frm (c)] (e) & (f) The QQ-plt fr lg-ratis and standardized ratis t remve the systematic biases befre carrying ut any statistical analysis The mdel () was applied t estimate the print-tip blck and intensity effects fr each array Fr the array given in Figures 5(a) and 5(b), the estimated functin m is depicted in Figure 5(c), in which unnrmalized and nrmalized lg-ratis are pltted against their assciated lg-intensities The estimated values f β and γ are nt reprted here The estimates were btained by the rdinary least-squares methd With the estimated blck and intensity effects, the systematic biases in 9,968 genes can be remved via (2) The nrmalized results are presented in Figure 5(d) t is clear that the lw intensity is assciated with high variatin in lg-ratis The degree f heterscadesticity (the cnditinal standard deviatin) is estimated by using the methd in Fan and Ya (998) The estimated standard deviatin functin ˆσ(X g ) is als pltted in Figure 5(d) 20

21 Let ˆµ( ) be the regressin functin f the nrmalized lg-ratis {Y g } n its assciated lg-intensities {X g } The quantile-quantile plts fr checking nrmality fr the nrmalized lg-ratis {Y g } and the standardized lg-ratis {(Y g ˆµ(X g ))/ˆσ(X g )} are depicted in Figures 5(e) and 5(f) As shwn in Figure 5(f), after standardizatin, the data becme mre nrmally distributed This indicates that the degree f heterscedasticity has been prperly assessed Further analysis f this data set can be fund in Fan et al(2004) 5 Discussin Mtivated by the prblem f the nrmalizatin f cdna micrarray data, tw semiparametric mdels are prpsed The interesting features f the mdels are that the number f nuisance parameters is prprtinal t the sample size These nuisance parameters can nt be estimated cnsistently Hwever, the parameters f main interest fr the nrmalizatin prblem can be estimated cnsistently The cst fr estimating the nuisance parameters is pinned dwn: each nuisance parameter csts us basically ne data pint This is the minimum price we have t pay, as we have demnstrated Our prpsed mdel has a wide spectrum f applicatins n additin t be applicable t varius situatins with within-array replicatins, it can even be applied t the case withut within-array replicatins t can als be used at blck level and this avids the additive assumptin n the intensity and blck effects Aggregatin is a pwerful apprach in imprving accuracy f estimated parameters As we have demnstrated, it reduces the price fr estimating ne nuisance parameter per data pint t per /J data pint per array As a result, the blck effects are estimated mre accurately n the nrmalizatin prcess, the main surce f estimatin errrs cme frm the nnparametric cmpnent the intensity effect Thus, as far as the asympttic is cncerned, aggregatin helps nly partially in the accuracy f nrmalizatin Hwever, as we have shwn in the simulatin, the aggregatin des help in estimating bth blck and intensity effects fr finite sample Aggregatin gives us much mre data pints fr remving systematic biases t allws us t impse sme mre flexible mdels t access the blck and intensity effects Because f increased sample size, we have mre flexibility in prpsing different kind f semiparametric mdels fr nrmalizatin and analysis f data Within-array replicatins are pwerful fr remving systematic biases and level f measurement errrs t is nt difficult t print several hundreds f repeated clnes in a cdna chip Thus, ur requirement that G is large shuld be easy t fulfill With increased sample size and ur analysis technique, the systematic biases due t the experimental variatins can be better remved 2

22 References Carrll, RJ, Fan, J, Gijbels,, and Wand, MP (997) Generalized partially linear single-index mdels Jur Ameri Satist Assc, 92, Graig, BA, Black, MA and Derge, RW (2003) Gene expressin data: the technlgy and statistical analysis Jurnal f Agricultural, Bilgical, and Envirnmental Statistics, 8, -28 Dudit, S, Yang, YH, Luu, P, Lin, DM, Peng, V, Ngai, Jand Speed, TP (2002) Nrmalizatin fr cdna micrarray data: a rbust cmpsite methd addressing single and multiple slide systematic variatin Nucleic Acids Research, 30, e5 Fan, J (992) Design-adaptive nnparametric regressin J Amer Statist Assc, 87, Fan, J and Gijbels, (995) Data-driven bandwidth selectin in lcal plynmial fitting: variable bandwidth and spatial adaptatin J Ryal Statist Sc B, 57, Fan, J, Tam, P, Vande Wude, G and Ren, Y (2004) Nrmalizatin and analysis f cdna micr-arrays using within-array replicatins applied t neurblastma cell respnse t a cytkine Prc Natl Acad Sci, 0, Fan, J and Ya, Q (998) Efficient estimatin f cnditinal variance functins in stchastic regressin Bimetrika, 85, Grlleau, A, Bwman, J, Pradet-Balade, B, Puravs E, Hanash, S, Garcia-Sanz, JA, and Beretta, L (2002) Glbal and specific translatinal cntrl by Rampamycin in T cells uncvered by micrarrays and prtemics Jur Bil Chem, 277, Härdle, W, Liang, H and Ga, J (2000) Partially Linear Mdels (in press) Springer-Verlag, Heidelberg Huang, J, Wang, D and Zhang, C (2003) A tw-way semi-linear mdel fr nrmalizatin and significant analysis f cdna micrarray data Manuscript Huang, J and Zhang, CH (2003) Asympttic analysis f a tw-way semiparametric regressin mdel fr micrarray data Technical reprt # , Rutgers University Krll, TC and Wölfl, S (2002) Ranking: a clser lk n glbalisatin methds fr nrmalisatin f gene expressin arrays Nucleic Acids Research, 30, e50 Mack, Y P and Silverman, B W (982) Weak and Strng unifrm cnsistency f kernel regressin and density estiamtes Z Wahr verw Geb, 6, Neyman, J and Sctt, E (948) Cnsistent estimates based n partially cnsistent bservatins Ecnmetrica, 6,

23 Opsmer, J D and Ruppert, D (997) Fitting a bivariate additive mdel by lcal plynmial regressin Ann Statist, 25, 86-2 Ra, C R and Tutenburg, H (999) Linear mdels: Least Squares and Alternatives 2nd ed, Springer, New Yrk Ruppert, D (997) Empirical-bias bandwidths fr lcal plynmial nnparametric regressin and density estimatin J Amer Statist Assc, 92, Speckman, P (988) Kernel smthing in partial linear mdels J Ryal Statist Sc B, 50, Tseng, GC, Oh, MK, Rhlin, L, Lia, JC and Wng, WH (200) ssues in cdna micrarray analysis: quality filtering, channel nrmalizatin, mdels f variatins and assessment f gene effects Nucleic Acids Research, 29, Appendix The fllwing technical cnditins are impsed They are nt weakest pssible cnditins, but they are impsed t facilitate the technical prfs () The functin m( ) has a bunded secnd derivative (2) Σ is nn-singular, and E(Z X = x) is Lipschitz cntinuus in x (3) Each cmpnent f Z is bunded (4) The randm variable X has a bunded supprt Ω ts density functin f( ) is Lipschitz cntinuus and bunded away frm 0 n Ω (5) The functin K( ) is a symmetric density functin with cmpact supprt (6) nh 8 0 and nh 2 /(lg n) 2 The fllwing ntatin will be used in the prfs f the lemmas and therems Let µ i = u i K(u)du, ν i = u i K 2 (u)du and c n = {lg(/h)/(nh)} 2 + h 2 Let D x = X X h X n X h, and S = [ 0]{D T x W x D x } D T x W x [ 0]{D T x n W xn D xn } D T x n W xn, where W x = diag{k h (X X),, K h (X n X)}, K( ) is a kernel functin, h is a bandwidth and K h ( ) = K( /h)/h Set Φ(X) = E(Z T X) 23

24 Lemma Let (X, Y ),, (X n, Y n ) be iid randm vectrs, where the Y i s are scalar randm variables Further assume that E Y i 4 < and sup x y 4 f(x, y)dy <, where f dentes the jint density f (X, Y ) Let K be a bunded psitive functin with a bunded supprt, and satisfies Lipschitz s cnditin Then, under Cnditin (6), [ sup Kh (X i X)Y i E{K X h (X i X)Y i } ] ( {lg(/h) } ) 2 = O p n nh i= Prf f Lemma : This fllws immediately frm the result f Mack and Silverman (982) Lemma 2 Under Cnditins ()-(6), we have n ZT n ( P B) Z n where Σ = E{Z E(Z X)} T {Z E(Z X)} Prf f Lemma 2: Nte that ( n D T i= x W x D x = K h(x i X) n i= X i X h K h (X i X) P Σ, ) n i= X i X h K h (X i X) n i= ( X i X h ) 2 K h (X i X) Each element f the abve matrix is in the frm f a kernel regressin By Lemma ( ) 0 n D T x W x D x = f(x) + O p (c n ) (A) 0 µ 2 By using the same argument, n D T x W x Z n = f(x)φ(x)(, 0) T + O p (c n ) Cmbining the last tw results yields that unifrmly in X Ω, [, 0]{D T x W x D x } D T x W x Z n = Φ(X) + O p (c n ) Then we have, and Z n = By the law f large numbers, we have SZ n = n ZT n Z n = n Z T Φ(X ) Z T n Φ(X n ) Φ(X ) Φ(X n ) + O p(c n ) + O p(c n ) ˆ=A + O p (c n ) {Z i Φ(X i ) T }{Z T i Φ(X i )} + O p (c n ) i= P E(Z E(Z X)(Z E(Z X) T = Σ (A2) 24

25 Hence, t prve the lemma, we nly cnsider the limit f n ZT n P Bn Zn t is easy t shw that n ZT n P Bn Zn = n A T P Bn A + O p (c n ) Let (P Bn ) ij ˆ=p ij and (A) ij ˆ=a ij = Z ij E(Z ij X i ), where Z ij is the jth cmpnent f randm vectr Z i, which represents the ith bservatin f Z Then, the (i, j) cmpnent f n AT P Bn A is ( n AT P Bn A) ij = n Fr the term 2, we have k= l= E 2 2 = n 2 E { a ki p kl a lj = n k l a ki p kk a kj + n k= a ki p kl a lj ˆ= + 2 k l k 2 l 2 a k ip k l a l ja k2 ip k2 l 2 a l2 j Nte that (Z, X ), (Z n, X n ) are iid, and that p ij depends nly n {X,, X n } fr any pair (i, j) Since E(a k j X k ) = 0, we have } E{a k ip k l a l ja k2 ip k2 l 2 a l2 j} = E{p k l p k2 l 2 E(a k ia l ja k2 ia l2 j X k, X k2, X l, X l2 )} = E{p k l p k2 l 2 E(a l ja k2 ia l2 j X k2, X l, X l2 )E(a k i X k )} = 0, when k k 2 and k l 2 Using the same argument and p kl = p lk, we have E 2 2 = n 2 E{a ki p kl a lj } 2 + n 2 k l E{p 2 kl a kia kj a lj a li } k l Since a ij s are unifrmly bunded by Cnditin (3), where C is a cnstant Hence, E 2 2 2C n 2 k l p 2 kl 2C n 2 tr(p2 Bn ) 2C n 2 = p () (A3) Nte that can be decmpsed as = p kk (a ki a kj Ea ki a kj ) + n n k= p kk Ea ki a kj ˆ=J + J 2 Since tr(s) = O p (/h) and tr(ss T ) = O p (/h), it is easy t knw that tr(p Bn ) = n + O p(/h) k= Hence J 2 = Σ ij + O p ( nh ), where Σ ij is the (i, j)th element f Σ Furthermre, if we can shw that p kk + O p( nh ), (A4) (A5) 25