Method of Moments Estimation in Linear Regression with Errors in both Variables J.W. Gillard and T.C. Iles

Transcription

1 Method of Moment Etimation in Linear Regreion with Error in both Variable by J.W. Gillard and T.C. Ile Cardiff Univerity School of Mathematic Technical Paper October 005 Cardiff Univerity School of Mathematic, Senghennydd Road, Cardiff, CF4 4AG

2 Content 1. Introduction 3. Literature Survey 6 3. Statitical aumption 9 4. Firt and econd order moment equation 1 5. Etimator baed on the firt and econd order moment Etimator making ue of third order moment Etimator making ue of fourth order moment 1 8. Variance and covariance of the etimator 4 9. Dicuion A ytematic approach for fitting line with error in both variable Reference Appendice Figure 43

3 1. Introduction The problem of fitting a traight line to bivariate (x, y) data where the data are cattered about the line i a fundamental one in tatitic. Method of fitting a line are decribed in many tatitic text book, for example Draper and Smith (1998) and Kleinbaum et al (1997). The uual way of fitting a line i to ue the principle of leat quare, finding the line that ha the minimum um of the quare of ditance of the point to the line in the vertical y direction. Thi line i called the regreion line of y on x. In the jutification of the choice of thi line it i aumed that deviation of the obervation from the line are caued by unexplained random variation that i aociated with the variable y. Implicitly it i aumed that the variable x i meaured without error or other variation. Clearly, if it i felt that deviation from the line are due to variation in x alone the appropriate method would be to ue the regreion line of x on y, minimiing the um of quare in the horizontal direction. The random deviation of the obervation from the uppoed underlying linear relationhip are uually called the error. Although the word error i a very common term it i an unfortunate choice of word; the variation may incorporate not jut meaurement error but any other ource of unexplained variation that reult in catter from the line. Some author have uggeted that other term might be ued, diturbance, departure, perturbation, noie and random component being amongt the uggetion. In thi report, however, becaue of the wide ue of the word, the variation from the line will be decribed a error. In many invetigation the catter of the obervation arie becaue of error in both meaurement. Thi problem i known by many name, the commonet being error in variable regreion and meaurement error model. The former name i ued throughout thi report. Caella and Berger (1990) wrote of thi problem, '(it) i o different from imple linear regreion... that it i bet thought of a a completely different topic'. There i a very extenive literature on the ubject, but publihed work i mainly in the form of article in the technical journal, mot of which deal with a particular apect of the problem. Relatively few tandard text book on regreion theory contain comprehenive decription of olution to the problem. A brief literature urvey i given in the next ection. 3

4 We believe that the error in variable regreion problem i potentially of wide practical application in the analyi of experimental data. One of the aim of thi report therefore i to give ome guidance for practitioner in deciding how an error in variable traight line hould be fitted. We give imple formula that a practitioner can ue to etimate the lope and intercept of an optimum line together with variance term that are alo included in the model. Very few previou author have given formula for the tandard error of thee etimator, and we offer ome advice regarding thee. Indeed, a detailed expoition on the variance covariance matrice for mot of the etimator in thi report i included in Gillard and Ile (006). In our approach we make a few aumption a are neceary to obtain etimator that are reliable. We have found that traightforward etimator of the parameter and their aymptotic variance can be found uing the method of moment principle. Thi approach ha the advantage of being imple to follow for reader who are not principally intereted in the methodology itelf. The method of moment technique i decribed in many book of mathematical tatitic, for example Caella and Berger (1990), although here, a elewhere, the treatment i brief. In common with many other mathematical tatitical text, they gave greater attention to the method of maximum likelihood. Bowman and Shenton (1988) wrote that 'the method of moment ha a long hitory, involve an enormou literature, ha been through period of evere turmoil aociated with it ampling propertie compared to other etimation procedure, yet urvive a an effective tool, eaily implemented and of wide generality'. Method of moment etimator can be criticied becaue they are not uniquely defined, o that if the method i ued it i neceary to chooe amongt poible etimator to find one that bet uit the data being analyed. Thi prove to be the cae when the method i ued in error in variable regreion theory. Neverthele the method of moment ha the advantage of implicity, and alo that the only aumption that have to be made are that low order moment of the ditribution decribing the obervation exit. We alo aume here that thee ditribution are mutually uncorrelated. It i relatively eay to work out the theoretical aymptotic variance and covariance of the etimator uing the delta method outlined by Cramer (1946). The information in thi report will enable a practitioner to fit the line and calculate approximate confidence interval for the 4

5 aociated parameter. Significance tet can alo be done. A limitation of the formula i that they are aymptotic reult, o they hould only be ued for moderate or large data et. 5

6 . Literature Survey A mentioned above, the error in variable regreion problem i rarely included in tatitical text. There are two text devoted entirely to the error in variable regreion problem, Fuller (1987) and Cheng and van Ne (1999). Caella and Berger (1990) ha an informative ection on the topic, Sprent (1969) contain chapter on the problem, a do Kendall and Stuart (1979) and Dunn (004). Draper and Smith (1998) on the other hand, in their book on regreion analyi, devoted only 7 out of a total of almot 700 page to error in variable regreion. The problem i more frequently decribed in Econometric text, for example Judge et al (1980). In thee text the method of intrumental variable i often given prominence. Intrumental variable are uncorrelated with the error ditribution, but are highly correlated with the predictor variable. The extra information that thee variable contain enable a method of etimating the parameter of the line to be obtained. Carroll et al (1995) decribed error in variable model for non-linear regreion, and Seber and Wild (1989) included a chapter on thi topic. Probably the earliet work decribing a method that i appropriate for the error in variable problem wa publihed by Adcock (1878). He uggeted that a line be fitted by minimiing the um of quare of ditance between the point and the line in a direction perpendicular to the line, the method that ha come to be known a orthogonal regreion. Kummel (1879) took the idea further, generaliing to a line that ha minimum um of quare of ditance of the obervation from the line in a direction other than perpendicular. Pearon (1901) generalied the error in variable model to that of multiple regreion, where there are two or more different x variable. He alo pointed out that the lope of the orthogonal regreion line i between thoe of the regreion line of y on x and that of x on y. The idea of orthogonal regreion wa included in Deming' book (1943), and orthogonal regreion i ometime referred to a Deming regreion. Another method of etimation that ha been ued in error in variable regreion i the method of moment. Geary (194, 1943, 1948 and 1949) wrote a erie of paper on the method, but uing cumulant rather than moment in the later paper. Drion (1951), in a paper that i infrequently cited, ued the method of moment, and gave 6

7 ome reult concerning the variance of the ample moment ued in the etimator that he uggeted. More recent work uing the moment approach ha been written by Pal (1980), van Montfort et al (1987), van Montfort (1989) and Cragg (1997). Much of thi work centre on a earch for optimal etimator uing etimator baed on higher moment. Dunn (004) gave formula for many of the etimator of the lope that we decribe later in thi report uing a method of moment approach. However, he did not give information about etimator baed on higher moment and it turn out that thee are the only moment baed etimator that can be ued unle there i ome information about the relationhip additional to the (x, y) obervation. Neither did he give information about the variance of the etimator. Another idea, firt decribed by Wald (1940) and taken further by Bartlett (1949), i to group the data, ordered by the true value of the predictor variable, and ue the mean of the group to obtain etimator of the lope. The intercept i then etimated by chooing the line that pae through the centroid ( x,y) of the complete data et. A difficulty of the method, noted by Wald himelf, i that the grouping of the data cannot, a may at firt be thought, be baed on the oberved value without making further aumption. In order to preerve the propertie of the random variable underlying the method it i neceary that the grouping be baed on ome external knowledge of the ordering of the data. In depending on thi extra information, Wald' grouping method i a pecial cae of an intrumental variable method, the intrumental variable in thi cae being the ordering of the true value. Gupta and Amanullah (1970) gave the firt four moment of the Wald etimator and Gibon and Jowett (1957) invetigated optimum way of grouping the obervation. Madanky (1959) reviewed ome apect of grouping method. Lindley (1947) and many ubequent author approached the problem of error in variable regreion from a likelihood perpective. Kendall and Stuart (1979), Chapter 9, reviewed the literature and outlined the likelihood approach. A diadvantage of the likelihood method in the error in variable problem i that it i only tractable if all of the ditribution decribing variation in the data are aumed to be Normal. In thi cae a unique olution i only poible if additional aumption are made concerning the parameter of the model, uually aumption about the error variance. 7

8 Neverthele, maximum likelihood etimator have certain optimal propertie and it i poible to work out the aymptotic variance-covariance matrix of the etimator. Thee were given for a range of aumption by Hood et al (1999). The likelihood approach wa alo ued by Dolby and Lipton (197), Dolby (1976) and Cox (1976) to invetigate the error in variable regreion problem where there are replicate meaured value at the ame true value of the predictor variable. Lindley and el Sayyad (1968) decribed a Bayeian approach to the error in variable regreion problem and concluded that in ome repect the likelihood approach may be mileading. A decription of a Bayeian approach to the problem, with a critical comparion with the likelihood method, i given by Zellner (1980). Golub and van Loan (1980), van Huffel and Vanderwalle (1991) and van Huffel and Lemmerling (00) have developed a theory that they have called total leat quare. Thi method allow the fitting of linear model where there are error in the predictor variable a well a the dependent variable. Thee model include the linear regreion one. The idea i linked with that of adjuted leat quare, that ha been developed by Kukuh et al (003) and Markovky et al (00, 003). Error in variable regreion ha ome imilaritie with factor analyi, a method in multivariate analyi decribed by Lawley and Maxwell (1971) and Johnon and Wichern (199) and elewhere. Factor analyi i one of a family, called latent variable method (Skrondal and Rabe-Heketh, 004), that include the error in variable regreion problem. Dunn and Robert (1999) ued a latent variable approach in an error in variable regreion etting, and more recently extenion combining latent variable and generalied linear model method have been devied (Rabe-Heketh et al, 000, 001). Over the year everal author have written review article on error in variable regreion. Thee include Kendall (1951), Durbin (1954), Madanky (1959), Moran (1971) and Anderon (1984). Rigg et al (1978) performed imulation exercie comparing ome of the lope etimator that have been decribed in the literature. 8

9 3. Statitical Aumption The notation in the literature for the error in variable regreion problem differ from author to author. In thi report we ue a notation that i imilar to that ued by Cheng and van Ne (1999), and that appear to be finding favour with other modern author. It i, unfortunately, different from that ued by Kendall and Stuart (1979), and ubequently adopted by Hood (1998) and Hood et al (1999). We uppoe that there are n individual in the ample with true value (ξ i, η i ) and oberved value (x i, y i ). It i believed that there i a linear relationhip between the two variable ξ and η. η i = α + βξ i (1) However, there i variation in both variable that reult in a deviation of the obervation (x i, y i ) from the true value (ξ i, η i ) reulting in a catter about the traight line. Thi catter i repreented by the addition of random error repreenting the variation of the oberved from the true value. x i = ξ i + i () y i = η i + i = α + βξ i + i (3) The error i and i are aumed to have zero mean and variance that do not change with the uffix i. E[ i ] = 0, Var[ i ] = E[ i ] = 0, Var[ i ] = We aume that higher moment alo exit. E[ 3 i. ] = µ 3, E[ ] = µ 4 E[ 3 i ] = µ 3, E[ 4 i ] = µ 4. 4 i We alo aume that the error are mutually uncorrelated and that the error i are uncorrelated with i. E[ i j ] = 0, E[ i j ] = 0 (i j) E[ i j ] = 0 for all i and j (including i = j). 9

10 Some author have treed the importance of a concept known a equation error. Further detail are given by Fuller (1987) and Carroll and Ruppert (1996). Equation error introduce an extra term on the right hand ide of equation (3). y i = η i + ω i + i = α + βξ i +ω i + i Dunn (004) decribed the additional equation error term ω i a '(a) new random component (that) i not necearily a meaurement error but i a part of y that i not related to the contruct or characteritic being meaured'. It i not intended to model a mitake in the choice of equation to decribe the underlying relationhip between ξ and η. Auming that the equation error term have a variance ω that doe not change with i and that they are uncorrelated with the other random variable in the model the practical effect of the incluion of the extra term i to increae the apparent variance of y by the addition of ω. We do not conider in thi report method for ue where there may be expected to be erial correlation amongt the obervation. Sprent (1969) included a ection on thi topic and Karni and Weiman (1974) ued a method of moment approach, making ue of the firt difference of the obervation, auming that a non zero autocorrelation i preent in the erie of obervation. In much of the literature on error in variable regreion a ditinction i drawn between the cae where the ξ i are aumed to be fixed, albeit unknown, quantitie and the cae where ξ i are aumed to be a random ample from a population. The former i known a the functional and the latter the tructural model. Caella and Berger (1990) decribed the theoretical difference in thee two type of model. Uing the approach adopted in thi report it i not neceary to make the ditinction. All that i aumed i that the ξ are mutually uncorrelated, are uncorrelated with the error and that the low order moment exit. Neither the problem of etimation of each individual ξ i in the functional model nor the problem of predicting y i invetigated in thi report. Whether the ξ are aumed to be fixed or a random ample we find only etimator for the low order moment. 10

11 The aumption that we make about the variable ξ are a follow. E[ξ i ] = µ, Var[ξ i ] =. In ome of the work that i decribed later the exitence of higher moment of ξ i alo aumed. E[(ξ i - µ) 3 ] = µ ξ3, E[(ξ i - µ) 4 ] = µ ξ4 The variable ξ i are aumed to be mutually uncorrelated and uncorrelated with the error term and. E[(ξ i - µ)(ξ j - µ)] = 0 (i j) E[(ξ i - µ) j ] = 0 and E[(ξ i - µ) j ] = 0 for all i and j. In order to etimate variance and covariance it i neceary later in thi report to aume the exitence of moment of ξ of order higher than the fourth. The rth moment r i denoted byµ = E[( ξ µ )]. ξr i 11

12 4. Firt and Second Order Moment Equation The firt order ample moment are denoted by x = and y =. n n The econd order moment are notated by (x x)(y y) i i xy =. n x i (x x) i xx =, n y i (y y) i yy = and n No mall ample correction for bia i made, for example by uing (n - 1) a a divior for the variance rather than n. Thi i becaue the reult on variance and covariance that we give later on in the report are reliable only for moderately large ample ize, generally 50 or more, where the adjutment for bia i negligible. Moreover, the algebra needed for the mall ample adjutment complicate the formula omewhat. The moment equation in the error in variable etting are given in the equation below. A tilde i placed over the ymbol for a parameter to denote the method of moment etimator. We have ued thi ymbol in preference to the circumflex, often ued for etimator, to ditinguih between method of moment and maximum likelihood etimator. Firt order moment: x =µ (4) y =α+βµ (5) Second order moment: xx yy xy = + (6) =β + (7) (8) =β It can readily be een from equation (6), (7) and (8) that there i a hyperbolic relationhip between method of moment etimator and of the error variance. Thi wa called the Frich hyperbola by van Montfort (1989). 1

13 ( )( ) = ( ) (9) xx yy xy Thi i a ueful equation in that it relate pair of etimator of and that atify equation (6), (7) and (8). The potential application of the Frich hyperbola are dicued further in Section 9. One of the difficultie with the error in variable regreion problem i apparent from an examination of equation (4) - (8). There i an identifiability problem if thee equation alone are ued to find etimator. There are five moment equation of firt or econd order but there are ix unknown parameter. It i therefore not poible to olve the equation to find unique olution without making additional aumption. One poibility i to ue higher moment, and thi i decribed later in the report. Another poibility i to ue additional information in the form of an intrumental variable. A third poibility, and the one that i invetigated firt, i to aume that there i ome prior knowledge of the parameter that enable a retriction to be impoed. Thi then allow the five equation to be olved. There i a comparion with thi identifiability problem and the maximum likelihood approach. In thi approach, the only tractable aumption i that the ditribution of i, i and ξ i, are all Normal. Thi in turn lead to the bivariate random variable (x, y) having a bivariate Normal ditribution. Thi ditribution ha five parameter, and the maximum likelihood etimator for thee parameter are identical to the method of moment etimator baed on the five firt and econd moment equation. In thi cae therefore it i not poible to find olution to the likelihood equation without making an additional aumption, reticting the parameter pace. The retriction that we decribe in Section 5 below are one that have been ued by previou author uing the likelihood method. The likelihood function for any other ditribution than the Normal i complicated and the method i difficult to apply. However the method of moment approach uing higher moment and without auming a retiction in the parameter pace, can be ued without making the aumption of Normality. 13

14 5. Etimator Baed on the Firt and Second Moment So that etimating equation tand out from other numbered equation, they are marked by an aterik. Equation (1) give the etimator for µ directly µ=x (10)* The etimator for all the remaining parameter are eaily expreed in term of the etimator β of the lope. Equation (4) and (5) can be ued to give an equation for the intercept α in term of β. α= y β x (11)* Thu the fitted line in the (x, y) plane pae through the centroid ( x, y) of the data, a feature that i hared by the imple linear regreion equation. Equation (8) yield an equation for, with β alway having the ame ign a xy. = β xy (1)* If the error variance i unknown, it i etimated from equation (6). = (13)* xx Finally if i unknown, it i etimated from equation (7) and the etimator for β. = (14)* yy β Since variance are never negative there are retriction on permiible parameter value, depending on the value taken by the ample econd moment. Thee 14

15 condition are often called admiibility condition. The traightforward condition, enabling non negative variance etimate to be obtained are given below. xx > yy > Alone, thee condition are not ufficient to enure that the variance etimator are non negative. The error in variable lope etimator mut lie between the y on x and x on y lope etimator xy xx and yy xy repectively. Other admiibility condition, relevant in pecial cae, are given in Table 1. Admiibility condition are dicued in detail by Kendall and Stuart (1979), Hood (1998), Hood et al (1999) and Dunn (004). We now turn to the quetion of the etimation of the lope. There i no ingle etimator for the lope that can be ued in all cae in error in variable regreion. Each of the retriction aumed on the parameter pace to to get around the identifiability problem dicued above i aociated with it own etimator of the lope. In order to ue an etimator baed on the firt and econd order moment alone it i neceary for the practitioner to decide on the bai of knowledge of the invetigation being undertaken which retriction i likely to uit the purpoe bet. Table 1 ummarie the implet etimator of the lope parameter β derived by auming a retriction on the parameter. With one exception thee etimator have been decribed previouly; mot were given by Kendall and Stuart (1979), Hood et al (1999) and, in a method of moment context, by Dunn (1989). 15

16 Table 1: Etimator of the lope parameter b baed on firt and econd moment Retriction Intercept α known Etimator y α β 1 = x Admiibility Condition x 0 Variance Variance known known Reliability ratio κ= known + Variance ratio λ= known β = xy xx β = 3 yy xy xy β 4 = κ xx { } 1/ ( λ ) + ( λ ) + 4 λ( ) yy xx yy xx xy β 5 = xy xx > ( ) xy yy > xx yy > ( ) xy xx > yy None None λ ν= known ( ν 1) + ign( ){( ν 1) ( ) + 4ν } 1/ β xy xy xy xx yy β = 6 ν xx xx 0 Both variance and known. yy β = ign( ) 7 xy xx 1/ xx yy > > There i an ambiguity in the ign to be ued in the equation for β 6 and β 7. Thi i reolved by auming that the lope etimator alway ha the ame ign a xy, a mentioned above to enure that equation (11)* give a non negative etimate of the variance. A dicuion of thee etimator i given in Section 9. 16

17 It may eem that the retriction leading to the etimator β 6 i not one that would often be made on the bai of a priori evidence. The reaon for the incluion of thi etimator, which eem not to have been previouly uggeted, i that it i a generaliation of an etimator that ha been widely recommended, the geometric mean etimator. Thi i the geometric mean of the lope of the regreion of y on x and the reciprocal of the regreion of x on y. Section 9 contain further dicuion. Aymptotic variance concerning thi etimator will not be included in thi report. The aumption that both error variance and are known i omewhat different from the other cae. By auming that two parameter are known there are only four remaining unknown parameter, but five firt and econd moment equation that could be ued to etimate them. One poibility of obtaining a olution i to ue only four of the five equation (4) to (8) incluive, or a imple combination of thee. If equation (6) i excluded, the etimator for the lope β i β 3, but then the aumed value of will almot certainly not agree exactly with the value that would be obtained from equation (1)*. If equation (7) i excluded, the etimator for the lope i β, but then it i mot unlikely that the aumed value of will agree exactly with the value obtained from equation (13)*. If equation (6) and (7) are combined, uing the known ratio λ=, the etimator β 5 i obtained, and then neither of equation (1)* and (13)* will be atified by the a priori value aumed for and. Another poibility that lead to a imple etimator for the lope β i to exclude equation (8), and it i thi that lead to the etimator β 7 in Table 1. 17

18 6. Etimate Making Ue of the Third Moment The third order moment are written a follow. xxx xxy xyy (xi x) = n = 3 (xi x) (yi y) n (xi x)(yi y) = n (y y) 3 i yyy =. n The four third moment equation take a imple form. Some detail on the derivation of thee expreion i given in Appendix 1. (15) xxx =µ ξ3 +µ 3 =βµ (16) xxy ξ3 =βµ (17) xyy ξ3 (18) 3 yyy =βµ ξ3+µ 3 Together with the firt and econd moment equation, equation (4) - (8) incluive, there are now nine equation in nine unknown parameter. The additional parameter introduced here are the third moment µ ξ3, µ 3 and µ 3. There are therefore unique etimator for all nine parameter. However, it i unlikely in practice that there i a much interet in thee third moment a there i in the firt and econd moment, more epecially, the lope and intercept of the line. Thu a impler way of proceeding i probably of more general value. The implet way of making ue of thee equation i to make a ingle further aumption, namely that µ ξ3 i non zero. There i a practical requirement aociated with thi aumption, and thi i that the ample third moment hould be ignificantly different from 0. It i thi requirement that ha probably led to the ue of third moment etimator receiving relatively little attention in recent literature. It i not 18

19 alway the cae that the oberved value of x and y are ufficiently kewed to allow thee equation to be ued with any degree of confidence. Moreover ample ize needed to identify third order moment with a practically ueful degree of preciion are omewhat larger than i the cae for firt and econd order moment. However, if the aumption can be jutified from the data then a traightforward etimator for the lope parameter i obtained without auming anything known a priori about the value taken by any of the parameter. Thi etimator i obtained by dividing equation (17) by equation (16). xyy β 8 = (19)* xxy The value for β obtained from thi equation can then be ubtituted in equation (11)* - (14)* to etimate the intercept α and all three variance moment µ ξ3 can be etimated from equation (16)., and. The third µ = xxy ξ3 β 8 (0)* Etimator for µ 3 and µ 3 may be obtained from equation (15) and (18) repectfully. Other imple way of etimating the lope are obtained if the additional aumption µ 3 = 0 and µ 3 = 0 are made. Thee would be appropriate aumption to make if the ditribution of the error term and are ymmetric. Note, however, that thi doe not imply that the ditribution of ξ i ymmetric. The obervation have to be kewed to allow the ue of etimator baed on the third moment. With thee aumption the lope β could be etimated by dividing equation (16) by (15) or by dividing equation (18) and (17). β= β= xxy xxx yyy xyy 19

20 We do not invetigate thee etimator further in thi report, ince we feel that etimator that make fewet aumption are likely to be of the mot practical value. 0

21 7. Etimate Making Ue of the Fourth Moment The fourth order moment are written a xxxx xxxy xxyy xyyy yyyy (xi x) = n = 4 3 (xi x) (yi y) n (xi x) (y y) = n i (xi x)(yi y) = n (yi y) = n 4 3 By uing a imilar approach to the one adopted in deriving the third moment etimating equation, the fourth moment equation can be derived. =µ + 6 +µ (1) xxxx ξ4 4 =βµ + 3β () xxxy ξ4 (3) xxyy =βµ ξ4 +β ++ =βµ + 3β (4) 3 xyyy ξ4 =βµ + 6β+µ (5) 4 yyyy ξ4 4 Together with the firt and econd moment equation thee form a et of ten equation, but there are only nine unknown parameter. The fourth moment equation have introduced three additional parameter µ ξ4 µ 4 and µ 4, but four new equation. One of the equation i therefore not needed. The eaiet practical way of etimating the parameter i to ue equation () and (4), together with equation (6), (7) and (8). Equation () i multiplied by β and ubtracted from equation (4). 1

22 β = 3 β ( β ) xxxy xyyy Equation (6) i multiplied by β and ubtracted from equation (7). β =β xx yy Thu, making ue alo of equation (8) an etimating equation i obtained for the lope parameter β. 1/ 3 xyyy xy yy β 9 = (6)* xxxy 3xyxx There may be a practical difficulty aociated with the ue of equation (6)* if the random variable ξ i Normally ditributed. In thi cae the fourth moment i equal to 3 time the quare of the variance. A random variable for which thi property doe not hold i aid to be kurtotic. A cale invariant meaure of kurtoi i given by the following expreion µ γ = (7) If the ditribution of ξ ha zero meaure of kurtoi the average value of the five ample moment ued in equation (6)* are a follow. 3 4 E[ xyyy ] = 3β + 3β 4 E[ xxxy] = 3β + 3β E[ xx] yy = + E[ ] =β + E[ ]=β xy Then it can be een that the average value of the numerator of equation (6)* i approximately equal to zero, a i the average value of the denominator. Thu there i

23 an additional aumption that ha to be made for thi equation to be reliable a an etimator, and that i that equation (7) doe not hold, µ ξ4 mut be different from 3 4. In practical term, both the numerator and the denominator of the right hand ide of equation (6)* mut be ignificantly different from zero. If a reliable etimate of the lope β can be obtained from equation (6)*, equation (10)* - (13)* enable the intercept α and the variance, and to be etimated. The fourth moment µ ξ4 of ξ can then be etimated from equation (), and the fourth moment µ 4 and µ 4 of the error term and can be etimated from equation (0) and (4) repectively, though etimate of thee higher moment of the error term are le likely to be of practical value. Although β 9 ha a compact cloed form, it variance i rather cumberome. Indeed, the variance of β 9 depend on the ixth central moment of ξ. Since it i impractical to etimate thi moment with any degree of accuracy, there will be no dicuion of the aymptotic variance of thi etimator. 3

24 8. Variance and Covariance of the Etimator In order to derive formula for the aymptotic variance and covariance of the etimator derived in previou ection, the variance and covariance of the ample moment are needed. Further detail on variance and covariance of the etimator are included in the technical paper by Gillard and Ile (006). However, a brief expoition i given here, and in Appendix. Since mot of the etimator decribed in thi report are non linear function of the ample moment, the problem of finding exact formula for the variance and covariance i not a traightforward one. However, an approximate method, called the delta method, or method of tatitical differential, give imple formula for the etimator quoted here. Thee have proved in imulation tudie to be highly reliable even for moderate (n = 50) ample ize. The method i ometime decribed in tatitic text, for example DeGroot (1989) and i often ued in linear model to derive a variance tabiliation tranformation (ee Draper and Smith, 1998). For further detail ee Kotz and Johnon (1988) and Bihop et al (1975). The method i ued to approximate the expectation, and hence alo the variance and covariance, of function of random variable by making ue of a Taylor erie expanion about the expected value. λ For each of the retricted cae dicued earlier, (apart from the retriction υ= ) β the variance covariance matrice can be partitioned into a um of three matrice, A, B and C. Thi i reported in Gillard and Ile (006). The matrix A alone i needed if the aumption are made that ξ, and all have zero third moment and zero meaure of kurtoi, a given by equation (7). Thee aumption would be valid if all three of thee variable are Normally ditributed. The matrix B give the additional term that are neceary if ξ ha non zero third moment and meaure of kurtoi. It can be een that in mot cae the B matrice are pare, needing adjutment only for the term for Var[ ] and Cov[ µ, ]. The 4

25 exception are in the cae where the reliability ratio i known where the lope i etimated by β 4. The C matrice are additional term that are needed if the third moment and meaure of kurtoi are non zero for the error term and. We believe it to be likely that thee C matrice will prove of le value to practitioner than the A and B matrice. For etimator baed on higher order moment, the algebra i more cumberome, and the expreion are not a concie a for the retricted cae. In Gillard and Ile (006), the tool need to contruct the variance covariance matrice for β 8 are included. The expreion that are of mot practical ue in the application of regreion method are the variance and covariance of the etimator of the intercept and lope parameter α and β. Thee enable approximate tet and confidence interval to be calculated for thee parameter, and alo approximate confidence band to be found for the line. We give here the formula derived from the A and B matrice, that i auming that the error variable have Normal like third and fourth moment. The variance for the lope etimator in Table 1 baed on the firt and econd moment are firt given in Table. The formula for the variance of β 8 i not a imple, and i given later. Etimate of a combination of the parameter i needed in ome of the variance and covariance formula given below. Thi etimator i derived here. With the aumption that have been made up to thi point, the ditribution of the bivariate random variable (x, y) T ha a mean vector that i equal to (µ,α + βµ) T and variance covariance matrix given by the following expreion.. + Σ= β ξ ξ βξ β ξ + (8) 5

26 Thi variance covariance matrix i etimated by the matrix S. xx xy S = xy yy (9) The determinant of the matrix Σ i etimated by the determinant of S. Σ =β + +. Thi i therefore xx yy xy (30)* Σ = S = ( ) Etimator of the variance, and are given in equation (11)*, (1)* and (13)* repectively, and the fourth moment µ ξ4 i etimated from equation (). 6

27 Table : Variance and Etimator of the Variance for the Slope Parameter Etimator given in Table 1. Etimator Variance of lope b Etimator of variance of lope b β 1 β + nµ yy nx β 1 n 4 Σ+β 4 1 n( ) S + ( β ) β 3 1 n β 4 Σ n( ) S + β 3 β 4 1 n 4 4 Σ + (1 κ) β ( µ ξ4 3 ) 1 n( ) S + (1 κ) β 4( µ ξ4 3( )) β 5 Σ n 4 S n( ) β 7 1 ( β ) n β Σ (yy β 7 xx ) S + n( ) β 7 Notice that in the pecial cae that µ ξ4 = 3 the formula for the variance of 4 β i identical to that for β 5. Thi would hold if ξ i Normally ditributed. The above variance etimator aume that both and are Normally ditributed (or have Normal like third and fourth moment). Detail on the correction when and are not Normally ditributed are offered in Gillard and Ile (006). 7

28 We now give formula for the variance of the lope etimator β 8 baed on the third and fourth moment where it i aumed that the error term and are aumed to be Normally ditributed. The expreion Var[ β 8 ] ha been calculated when and are aumed not to be Normally ditributed by Gillard and Ile (006). 1 3 Var[ β 8] = βµ ξ4( +β ) + ( + ) 3 ( ) 6 + β + n( µ ξ3) β Notice that the formula involve the third and fourth moment of ξ, but not higher moment. To etimate thi variance, all three parameter,, and have to be etimated uing equation (11)*, (1)* and (13)* repectively. The third moment µ ξ3 i etimated from equation ()*. The fourth moment µ ξ4 can be etimated from one of equation (), (3) or (4). The combination ( ) +β, ( + ) and ( β + ) are etimated by, xx and yy repectively. ( β xx+ yy β xy) 8

29 9. Dicuion It wa not our intention in writing thi report to advocate the complete abandoment of imple linear regreion method. If there are no meaurement error aociated with the x variable in linear regreion, then = 0 and the optimum way of fitting the line if the ret of the aumption made in thi report hold true i to ue the leat quare regreion line of y on x. The intercept and lope etimator of thi line are both unbiaed. Converely, if there are no meaurement error aociated with the variable y then the regreion of x on y give the bet line. However, in the preence of meaurement error in both x and y, neither the x on y nor the y on x regreion give unbiaed etimator of the lope and intercept. In fact the true line lie between thee two extreme. (ee, for example, Caella and Berger, 1990). In thi report a number of poible olution have been preented to the problem of identifying an appropriate relationhip between the (unmeaured) variable ξ and η, baed on the obervation x and y. In the following ection a procedure i uggeted for electing the mot appropriate line for a particular purpoe. Some obervation on the inter-relationhip between thee different etimator i dicued here. The etimator that are given in Table 1 are, with one exception, maximum likelihood etimator for the cae where the variable ξ and the error and are all aumed to be Normally ditributed. The exception i β 7, the cae where both error variance are known, where the maximum likelihood etimator for the lope i β 5. Mot of thee cae were derived by Hood (1998) and Kendall and Stuart (1979). The derivation of the etimator in thi report i baed on the method of moment and no aumption of an exact form for thee ditribution i neceary. The cae where the intercept α i aumed to be known lead to the etimator β 1, which i a ratio of mean. In a ene, therefore, thi etimator i related to the ratio of mean etimator uggeted by Wald (1940) and Bartlett (1949) but baed on grouped data. 9

30 The etimator β, which hould be ued if prior knowledge give the value for the error variance of the error in the x variable, i a imple modification of the lope of the regreion of y on x. The modification i to ubtract the known variance from the um of quare of x, xx, in the denominator of the expreion. The effect of equation error have led ome author, notably Dunn (004), to recommend that an etimator be choen that relie only on information about. The difficulty of uing prior information of error variability in the y variable to etimate the variance i that uch information may underetimate the variance term on the right hand ide of equation (7), becaue the contribution made by the equation error term may be overlooked. Dunn' concluion i that etimator that aume prior knowledge of the error variance aociated with the meaurement of x, i more likely to be reliable than thoe that aume prior knowledge of Where it i believed that prior knowledge give a reliable value for the error variance of the error in the y variable the etimator β 3 hould be ued. Thi i a modification of the reciprocal of the lope of the regreion of x on y. The modification i to ubtract the known variance in the numerator of the expreion. from the um of quare of y, yy, Knowledge of the reliability ratio implicitly i knowledge of the bia of the y on x regreion lope where there are error in both variable. The unbiaed etimator β 4 in thi cae i obtained from the lope of the regreion line of y on x imply by dividing by the reliability ratio. The cae where the ratio λ of error variance i known, β 5, i related to the orthogonal regreion line. If λ = 1 thee ditance are in a direction that i orthogonal to the line. Caella and Berger (1990), amongt other author, gave a proof of thi reult. If λ 1 the line till ha a geometrical interpretation, but the um of quare of ditance from the line that i minimied i in a direction different from perpendicular. Rigg et al (1978), baed on their imulation tudie, recommended the ue of thi etimator 30

31 but emphaied the importance of having a reliable prior knowledge of the ratio λ of error variance. Ue of thi line ha been criticied on the ground that if the cale of meaurement of the line i changed then a different line would be fitted (Bland 000, p187). In fact, thi criticim cannot be ubtantiated a long a it i kept in mind that it i knowledge of the ratio of the error variance λ that i ued in fitting in line. If the data are recaled in any way there i an exactly correponding recaling of λ that lead to the ame line being fitted. The preence of equation error dicued above might, however, make it difficult to obtain a reliable a priori etimate of the ratio λ of error variance. The etimator β 6 ha been included in thi report becaue it i linked with other λ etimator. The ratio ν= i a dimenionle quantity. If ν = 0 the etimator β reduce to the regreion of x on y. If ν = it i the regreion of y on x. If ν = 1, the etimator reduce to a imple etimator. β= yy xx 1/ Thi i the geometric mean of the lope of the regreion of y on x, xy xx and the reciprocal of the lope of the regreion of x on y, yy xy. Probably becaue it i a compromie olution to the error in variable regreion problem and apparently make no ue of prior aumption about the parameter, it ha been recommended by everal author, for example Draper and Smith (1998, p 9). They gave a geometric interpretation of thi etimator. The line with thi lope ha the minimum um of product of the horizontal and vertical ditance of the obervation from the line. However, unle the retriction β = i true, the etimator i biaed. If however thi retriction i true the variance of etimator take a very imple form. Var[ β ] = Σ n( ξ +) 31

32 Thi i etimated by ( ) xx yy xy = xx xx S n( ) n( ) A technical criticim of the ue of thi etimator i that it may have infinite variance (Creay, 1956). A Creay pointed out, however, thi occur when the catter of the obervation in the (x, y) plane i o large in both direction that it i viually impoible to determine if one line or a different line at right angle hould be ued to decribe the relationhip between y and y. Thu the criticim applie in cae where a linear relationhip between x and y i not trongly indicated by the obervation. The geometric mean etimator i alo related to the orthogonal regreion etimator β 5. If the ratio λ i taken to be yy xx, the two etimator are identical. The etimator β 7 can clearly be een to be a modification of the geometric mean etimator decribed above in that both numerator and denominator are modified by ubtraction of the known error variance. If ( xx ) i ubtituted from equation (9), the Frich hyperbola, into the formula for β 7 the etimator β 3 i obtained. Similarly if ( yy ) i ubtituted from equation (9), the etimator β i obtained. In the cae where no knowledge i available that enable a retriction on the parameter pace to be aumed, none of the etimator baed on the firt and econd moment alone can be ued, although a dicued above the geometric mean etimator ha been uggeted a an ad hoc compromie. However, if the third order moment xyy and xxy are both ignificantly different from 0, the etimator β 8 can be ued, and i a reliable etimator if the ample ize i ufficiently large. Another poible olution that could be ued i β 9, but here the data mut be uch that both the numerator and denominator of the etimator are ignificantly different from zero. Doubtle there will be cae where there i inufficient detailed prior knowledge of the parameter of the model to aign precie value to parameter, but neverthele there may be ome range of value for thee parameter that are believed to be more 3

33 likely to be true than other. In uch cae there are two poible type of plot that may be of practical value in the identification of an appropriate range of value of the lope parameter β and hence, by uing equation (9)* to (13)* range of value for the other parameter. The firt of thee plot i the Frich hyperbola given in equation (9). An example of a Frich hyperbola plot of againt i given in Figure 1. The ue of thi plot can be illutrated by auming that it i believed that the error variance for x i about equal to 1.0, but i poibly between 0.8 and 1.. Thi give an error variance for y, that i approximately between 1. and 1.6, with 1.4 a the mot likely value. Thi range of indicated value for could then be appraied to determine whether thi range of value i plauible. The poibility of equation error mut alway be borne in mind, however. The indicated value of indicated by thi plot may at firt ight be larger than expected from meaurement error alone. It could be the equation error contribution that ha inflated the etimate. The econd type of plot that may be ueful in uch circumtance i a enitivity plot. Suppoe that the preferred etimator for the lope i the cae where the variance ratio λ i aumed to be known, but the precie value of λ i not known with certainty. The etimator of the lope in thi cae i β 5. { } λ + ( λ ) + 4 λ( ) yy xx yy xx xy β 5 = (33) xy A value of β can then be calculated for each plauible value of λ. A plot of uch value i given in Figure. Suppoe a priori evidence ugget that λ i equal to 1.4, but there i ome doubt about the exact value, and it i poible that λ might be between 1.33 and 1.5. The correponding value of β are between approximately and 1.155, with 1.15 a the mot likely value. Similar enitivity plot are readily devied for other etimator given in Table 1. In ome cae the etimator given in Table 1 will give imilar numerical value. One uch cae would be where the lope β i mall. In that cae the numerical value for β and β 5 are highly likely to be numerically imilar. Thi i becaue the correlation 33

34 between thee two etimator i approximately equal to 1 if β i mall. Uing the delta method decribed above an equation for thi correlation coefficient can be worked out. β Corr[ β, β 5] 1+ Σ Thu if β 7 i mall compared with Σ, or if 4 i mall compared with Σ thi correlation coefficient i approximately equal to 1, and in practice value of β and β 5 will be numerically imilar. In imilar fahion it can be hown that β 3 and β 5 will be numerically imilar if the lope β i large. 34

35 10. A Sytematic Approach for Fitting Line where there are Error in Both Variable. A ytematic procedure for etimating the lope of a traight line relationhip between y and x can now be preented, making ue of the theory preented in thi report. If the meaurement error in x i mall in comparion with that of y, ue the imple linear regreion of y on x. If the meaurement error in y i mall in comparion with that of x, ue the reciprocal of regreion of x on y. In the following cae it i aumed that there are believed to be ignificant meaurement error in both x and y. Equation for the etimator are given in Table 1, except a noted. Care ha to be taken for ome of thee etimator that admiibility condition are atified. In practice, if a variance etimate i obtained that i negative, the aumption that have been made are incorrect, and a reappraial of a priori information i needed. One may alo check that the lope etimate lie between the y on x regreion and x on y regreion etimate repectively. If the intercept α i known a priori, ue etimator β 1. If the error variance i known, ue etimator β. If the error variance i known, ue etimator β 3. If the reliability ratio κ= i known, ue etimator β 4. + If the ratio of error variance λ= i known, ue etimator β 5. 35

36 If no a priori knowledge i available, but the ample third moment are ignificantly different from zero, ue etimator β 8. For thi etimator to be reliable a ample ize of at leat 50 i needed. If no a priori knowledge i available, but the coefficient of kurtoi are ignificantly different from zero, ue etimator β 9. For thi etimator to be reliable a ample ize of at leat 100 i needed. If a ingle mot appropriate etimate for the lope i obtained, etimate of the variance, the intercept and the mean µ are obtained by uing equation (10)* to (14)* incluive. If imprecie prior information i available, and the condition for the ue of one of β 8 or β 9 are not atified, ue the Frich hyperbola and enitivity plot to identify a range of poible value for β that accord with the prior knowledge. If no a priori knowledge i available, and it i decided to make the ad hoc aumption that β =, or equivalently that the ratio of error variance λ i equal to yy xx, ue etimator β 6. Thi etimator hould be ued with ome care, however. It i unlikely that thee condition will be met in practice, and the etimator i then biaed. Alternatively, one might ue the orthogonal regreion etimator, β 6 with λ= 1. Thi i equivalent to minimiing the um of quare of the orthogonal projection from the data point to the regreion line. 36

37 11. Reference Adcock, R.J. (1878). A problem in leat quare. Analyt, 5, Anderon, T.W. (1984). Etimating linear tatitical relationhip. Ann. Statit., 1, Bartlett, M.S. (1949). Fitting a traight line when both variable are ubject to error. Biometric, 5, Bihop, Y.M.M., Fienberg, S., Holland, P. (1975). Dicrete Multivariate Analyi. MIT Pre, Cambridge. Bland, M. (000). An Introduction to Medical Statitic (3 rd edition). Oxford Univerity Pre, Oxford. Bowman, K.O. and Shenton, L.R. (1985). Method of moment. Encyclopedia of Statitical Science (Volume 5), , John Wiley & Son, Canada. Carroll, R.J. and Ruppert, D. (1996). The ue and miue of orthogonal regreion in linear error-in-variable model. Amer. Statit., 50, 1-6. Carroll, R.J., Ruppert, D., and Stefanki, L.A. (1995). Meaurement Error in Nonlinear Model. Chapman & Hall, London. Caella, G. and Berger, R.L. (1990). Statitical Inference. Wadworth & Brook/Cole, Pacific Grove, CA. Cheng, C-L. and Van Ne, J.W. (1999). Statitical Regreion with Meaurement Error. Arnold, London. Cox, N.R. (1976). The linear tructural relation for everal group of data. Biometrika, 63, Cragg, J.G. (1997). Uing higher moment to etimate the imple error-in-variable model. The RAND Journal of Economic, 8(0), S71-S91. Cramer, H. (1946). Mathematical Method of Statitic. Princeton Univerity Pre, Princeton, NJ. DeGroot, M.H. (1989). Probability and Statitic. Addion-Weley Publihing Company, USA. Deming, W.E. (1931). The application of leat quare. Philo. Mag. Ser. 7, 11, Draper, N. and Smith, H. (1998). Applied Regreion Analyi. Wiley, New York. 37

38 Drion, E.F. (1951). Etimation of the parameter of a traight line and of the variance of the variable, if they are both ubject to error. Indagatione Mathematicae, 13, Dolby, G.R. (1976). The ultratructural relation: a ynthei of the functional and tructural relation. Biometrika, 63, Dolby, G.R. and Lipton, S. (197). Maximum likelihood etimation of the general nonlinear functional relationhip with replicated obervation and correlated error. Biometrika, 59(1), Dunn, G. (004). Statitical Evaluation of Meaurement Error ( nd Edition). Arnold, London. Dunn, G. and Robert, C. (1999). Modelling method comparion data. Statitical Method in Medical Reearch, 8, Durbin, J. (1954). Error in variable. Int. Statit. Rev.,, 3-3. Fuller, W.A. (1987). Meaurement Error Model. Wiley, New York. Geary, R.C. (194). Inherent relation between random variable. Proc. R. Irih. Acad. Sect. A. 47, Gibon, W.M. and Jowett, G.H. (1957). Three-group regreion analyi. Part 1: Simple regreion analyi. Appl. Statit., 6, Gillard, J.W. and Ile, T.C. (006). Variance covariance matrice for linear regreion with error in both variable. Cardiff School of Mathematic Technical Report. Golub, G.H. and Van Loan, C.F. (1981). An analyi of the total leat quare problem. SIAM J. Numer. Anal., 17, Gupta, Y. P. and Amanullah (1970). A note on the moment of the Wald' etimator. Statitica Neerlandica Hood, K. (1998). Some tatitical apect of method comparion tudie. Cardiff Univerity Ph.D Thei. Hood, K., Nix, A. B. J., and Ile, T. C. (1999). Aymptotic information and variancecovariance matrice for the linear tructural model. The Statitician. 48(4), Johnon, R. A. and Wichern, D. W. (199). Applied Multivariate Statitical Analyi. Prentice-Hall, London. Judge, G.G., Griffith, W.E., Carter Hill, R. and Lee, T-C. (1980). The Theory and Practie of Econometric. Wiley, New York. 38

39 Karni, E. and Weiman, I. (1974). A conitent etimator of the lope in a regreion model with error in the variable. Journal of the American Statitical Aociation. 65, Kendall, M.G. (1951). Regreion, tructure, and functional relationhip, I. Biometrika, 38, Kendall, M.G. and Stuart, A. (1979). The Advanced Theory of Statitic, Vol. (4 th Edition). Griffin, London. Kleinbaum, D.G., Kupper, L.L., Muller, K.E. and Nizam, A (1997). Applied Regreion Analyi and Other Multivariable Method (3rd Edition). Duxbury Pre, CA. Kotz, S. and Johnon, N.L. (1988). Encyclopedia of Statitical Science. Wiley, New York. Kummel, C.H. (1879). Reduction of oberved equation which contain more than one oberved quantity. Analyt. 6, Kukuh, A., Markovky, I., and Van Huffel, S. (003). Conitent etimation in the bilinear multivariate error-in-variable model. Metrika, 57, Lawley, D. and Maxwell, A. (1971). Factor Analyi a a Statitical Method ( nd Edition). Elevier Publihing, New York. Lindley, D.V. (1947). Regreion line and the linear functional relationhip. J. R. Statit. Soc. Suppl., 9, Lindley, D.V. and El Sayyad, G.M. (1968). The Bayeian etimation of a linear functional relationhip. J. R. Statit. Soc. B, 30, Madanky, A. (1959). The fitting of traight line when both variable are ubject to error. J. Amer. Statit. Aoc., 54, Markovky I., Kukuh A., Van Huffel S., (00). Conitent leat quare fitting of ellipoid. Numeriche Mathematik. 98(1), Markovky I., Van Huffel S., Kukuh A., (003). On the computation of the multivariate tructured total leat quare etimator. Numer. Linear Algebra Appl. 11, Moran, P.A.P. (1971). Etimating tructural and functional relationhip. J. Multivariate Anal., 1, Pal, M. (1980). Conitent moment etimator of regreion coefficient in the preence of error in variable. J. Econometric, 14,

40 Pearon, K. (1901). On line and plane of cloet fit to ytem of point in pace. Philo. Mag., Rabe-Heketh, S., Pickle, A., and Taylor, C. (000). Generalied linear latent and mixed model. Stata Technical Bulletin, 53, Rabe-Heketh, S., Pickle, A., and Skrondal, A. (001). GLLAMM: A general cla of multilevel model and a Stata program. Multilevel Modelling Newletter, 13, Rigg, D. S., Guarnieri, J. A., and Addleman, S. (1978). Fitting traight line when both variable are ubject to error. Life Science., Seber, G.A.F. and Wild, C.J. (1989). Nonlinear Regreion. Wiley, New York. Skrondal, A. and Rabe-Heketh, S. (004). Generalied Latent Variable Modelling. Chapman and Hall/CRC, Florida. Sprent, P. (1969). Model in Regreion and Related Topic, Matheun & Co Ltd, London. Van Huffel, S. and Lemmerling, P. (Ed) (00). Total Leat Square and Error-in- Variable Modelling: Analyi, Algorithm and Application, Kluwer, Dordrecht. Van Huffel, S. (1997) and Vandewalle, J. (1991). The Total Leat Square Problem: Computational Apect and Analyi. SIAM, Philadelphia. Van Montfort, K. (1988). Etimating in Structural Model with Non-Normal Ditributed Variable: Some Alternative Approache. M & T Serie 1. DSWO Pre, Leiden. Van Montfort, K., Mooijaart, A., and de Leeuw, J. (1987). Regreion with error in variable. Statit. Neerlandica, 41, Wald, A. (1940). The fitting of traight line if both variable are ubject to error. Ann. Math. Statit., 11, Zellner, A. (1980). An Introduction to Bayeian Inference in Econometric. Wiley, New York. 40

41 1. Appendice Appendix 1 The moment equation baed on the third and fourth moment are lightly more difficult to derive than for the firt and econd order moment equation. One example illutrate the general approach. ( ) ( ) { ( i ) ( i )} ( i ) ( i ) E n = E x x y y xxy i i { } = E ξ ξ + βξ ξ+ 3 ( i ) ( i ) ( i ) ( i ) ( i ) = E βξ ξ + ξ ξ + βξ ξ ( i )( i )( i ) ( i )( i ) ( i ) ( i ) + βξ ξ + βξ ξ + Term of order n -1 are neglected, o the expectation of all the cro product in thi expreion are zero, becaue of the aumption that ξ, and are mutually uncorrelated and to order n -1 term uch a E ( ξ ξ i ) E n = n xxxy βµ. ξ3 are zero. Hence Appendix Suppoe etimator θ and φ of parameter θ and φ are calculated from two ample moment u and v. The formula below can readily be generalied for cae where three or four ample moment are ued in the etimator. θ= f(u,v) φ= g(u,v) Let f f = u u = u E[u], the partial derivative evaluated at the expected value of the ample moment, u. Then, f f f f Var θ Var[u] + Var[v] + Cov[u,v] u v u v f g f g f g f g Cov θφ, Var[ u] + Var[ v] + + Cov[ u,v]. u u v v u v v u 41

42 The algebra required to work out the variance and the covariance i quite lengthy. Neverthele the reulting formula are not difficult, and etimate of the parameter needed to etimate thee variance and covariance are readily obtained from Section 4, 5 and 6. 4

43 13. Figure Figure 1 Frich Hyperbola v Frich Hyperbola Figure 1.4 Plot Slope of m β againt v variance variance ratio λ ratio