

 Silvester Manning
 1 years ago
 Views:
Transcription
1 Applying MCMC Methods to Multilevel Models submitted by William J Browne for the degree of PhD of the University of Bath 1998 COPYRIGHT Attention is drawn tothefactthatcopyright of this thesis rests with its author This copy of the thesis has been supplied on the condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the prior written consent of the author This thesis may bemadeavailable for consultation within the University Library and may be photocopied or lent to other libraries for the purposes of consultation Signature of Author William J Browne
2 To Health, Happiness and Honesty
3 Summary Multilevel modelling and Markov chain Monte Carlo methods are two areas of statistics that have become increasingly popular recently due to improvements in computer capabilities, both in storage and speed of operation The aim of this thesis is to combine the two areas by tting multilevel models using Markov chain Monte Carlo (MCMC) methods This task has been split into three parts in this thesis Firstly the types of problems that are tted in multilevel modelling are identied and the existing maximum likelihood methods are investigated Secondly MCMC algorithms for these models are derived and nally these methods are compared to the maximum likelihood based methods both in terms of estimate bias and interval coverage properties Three main groups of multilevel models are considered Firstly N level Gaussian models, secondly binary response multilevel logistic regression models and nally Gaussian models with complex variation at level 1 Two simple 2 level Gaussian models are rstly considered and it is shown how to t these models using the Gibbs sampler Then extensive simulation studies are carried out to compare the Gibbs sampler method with maximum likelihood methods on these two models For the general N level Gaussian models, algorithms for the Gibbs sampler and two alternative hybrid Metropolis Gibbs methods are given and these three methods are then compared with each other One of the hybrid Metropolis Gibbs methods is adapted to t binary response multilevel models This method is then compared with two quasilikelihood methods via a simulation study on one binary response model where the quasilikelihood methods perform particularly badly All of the above models can also be tted using the Gibbs sampling method using the adaptive rejection algorithm in the BUGS package (Spiegelhalter et al 1994) Finally Gaussian models with complex variation at level 1 which cannot be tted in BUGS are considered Two methods based on Hastings update steps are given and are tested on some simple examples The MCMC methods in this thesis have been added to the multilevel modelling package MLwiN (Goldstein et al 1998) as a byproduct of this research
4 Acknowledgements I would rstly like to thank my supervisor, Dr David Draper whose research in the elds of hierarchical modelling and Bayesian statistics motivated this PhD I would also like to thank him for his advice and assistance throughout both my MSc and PhD I would like to thank my parents for supporting me both nancially and emotionally through my rst degree and beyond I would like to thank the multilevel models project team at the Institute of Education, in particular Jon Rasbash and Professor Harvey Goldstein for allowing me to work with them on the MLwiN package I would also like to thank them for their advice and assistance while I have been working on the package I would like to thank my brother Edward and his ance Meriel for arranging their wedding a month before I am scheduled to nish this thesis This way I can spread my worries between my PhD andmy best man's speech Iwould like to thank my girlfriends over the last three years for helping me through various parts of my PhD Thanks for giving me love and support when I needed it and making my life both happy and interesting I would like to thank the other members of the statistics group at Bath for teaching me all I know about statistics today I would like to thank my fellow oce mates, past and present for their humour, conversation and friendship and for joining me in my many pointless conversations Thanks to family and friends both in Bath and elsewhere Special thanks are due to the EPSRC for their nancial support \The only thing I know is that I don't know anything" Socrates
5 Contents 1 Introduction 1 11 Objectives 1 12 Summary of Thesis 2 2 Multi Level Models and MLn 4 21 Introduction JSP dataset 4 22 Analysing Redhill school data Linear regression Linear models 7 23 Analysing data on the four schools in the borough of Blackbridge ANOVA model ANCOVA model Combined regression Two level modelling Iterative generalised least squares Restricted iterative generalised least squares Fitting variance components models to the Blackbridge dataset Fitting variance components models to the JSP dataset Random slopes model Fitting models to pass/fail data Extending to multilevel modelling Summary 21 i
6 3 Markov Chain Monte Carlo Methods Background Bayesian inference Metropolis sampling Proposal distributions MetropolisHastings sampling Gibbs sampling Rejection sampling Adaptive rejection sampling Gibbs sampler as a special case of the MetropolisHastings algorithm Data summaries Measures of location Measures of spread Plots Convergence issues Length of burnin Mixing properties of Markov chains Multimodal models Summary Use of MCMC methods in multilevel modelling Example  Bivariate normal distribution Metropolis sampling MetropolisHasting sampling Gibbs sampling Results Summary 51 4 Gaussian Models 1  Introduction Introduction Prior distributions Informative priors Noninformative priors Priors for xed eects 55 ii
7 424 Priors for single variances Priors for variance matrices Level variance components model Gibbs sampling algorithm Simulation method Results : Bias Results : Coverage probabilities and interval widths Improving maximum likelihood method interval estimates for u Summary of results Random slopes regression model Gibbs sampling algorithm Simulation method Results Conclusions Simulation results Priors in MLwiN Gaussian Models 2  General Models General N level Gaussian hierarchical linear models Gibbs sampling approach Generalising to N levels Algorithm Computational considerations Method 2 : Metropolis Gibbs hybrid method with univariate updates Algorithm Choosing proposal distribution variances Adaptive Metropolis univariate normal proposals Method 3:Metropolis Gibbs hybrid method with block updates Algorithm Choosing proposal distribution variances Adaptive multivariate normal proposal distributions Summary Timing considerations 138 iii
8 6 Logistic Regression Models Introduction Multilevel binary response logistic regression models Metropolis Gibbs hybrid method with univariate updates Other existing methods Example 1 : Voting intentions dataset Background Model Results Substantive Conclusions Optimum proposal distributions Example 2 : Guatemalan child health dataset Background Model Original 25 datasets Simulating more datasets Conclusions Summary Gaussian Models 3  Complex Variation at level Model denition Updating methods for a scalar variance Metropolis algorithm for log Hastings algorithm for Example : Normal observations with an unknown variance Results Updating methods for a variance matrix Hastings algorithm with an inverse Wishart proposal Example : Bivariate normal observations with an unknown variance matrix Results Applying inverse Wishart updates to complex variation at level MCMC algorithm Example iv
9 743 Conclusions Method 2: Using truncated normal Hastings update steps Update steps at level 1 for JSP example Proposal distributions Example 2 : Nonpositive denite and incomplete variance matrices at level General algorithm for truncated normal proposal method Summary Conclusions and Further Work Conclusions MCMC options in the MLwiN package Further work Binomial responses Multinomial models Poisson responses for count data Extensions to complex variation at level Multivariate response models 188 v
10 List of Figures 21 Plot of the regression lines for the four schools in the Borough of Blackbridge Tree diagram for the Borough of Blackbridge Histogram of 1 using the Gibbs sampling method Kernel density plot of 1 using the Gibbs sampling method and a Gaussian kernel with a large value of the window width h Traces of parameter 1 and the running mean of 1 for a Metropolis run that converges after about 50 iterations Upper solid line in lower panel is running mean with rst 50 iterations discarded ACF and PACF for parameter 1 for a Gibbs sampling run of length 5000 that is mixing well and a Metropolis run that is not mixing very well Kernel density plot of 2 using the Gibbs sampling method and a Gaussian kernel Plots of the Raftery Lewis ^N values for various values of p, the proposal distribution standard deviation Plot of the MCMC diagnostic window in the package MLwiN for the parameter 1 from a random slopes regression model Plot of normal prior distributions over the range ({5,5) with mean 0andvariances 1,2,5,10 and 50 respectively Plots of biases obtained for the various methods against study design and parameter settings Trajectories plot of IGLS estimates for run of random slopes regression model where convergence is not achieved 85 vi
11 44 Plots of biases obtained for the various methods tting the random slopes regression model against value of u01 (Fixed eects parameters and level 1 variance parameter) Plots of biases obtained for the various methods tting the random slopes regression model against value of u01 (Level 2 variance parameters) Plots of biases obtained for the various methods tting the random slopes regression model against study design (Fixed eects parameters and level 1 variance parameter) Plots of biases obtained for the various methods tting the random slopes regression model against study design (Level 2 variance parameters) Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the variance components model on the JSP dataset Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 1 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the multivariate normal proposal distribution and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the multivariate normal proposal distribution and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 1 parameter in the random slopes regression model on the JSP dataset 131 vii
12 61 Plot of the eect of varying the scale factor for the univariate Normal proposal distribution rate on the Raftery Lewis diagnostic for the u 2 parameter in the voting intentions dataset Plots comparing the actual coverage of the four estimation methods with their nominal coverage for the parameters 0 1 and Plots comparing the actual coverage of the four estimation methods with their nominal coverage for the parameters 3 v 2 and u Plots of truncated univariate normal proposal distributions for a parameter, A is the current value, c and B is the proposed new value, M is max and m is min, the truncation points The distributions in (i) and (iii) have mean c, while the distributions in (ii) and (iv) have mean 175 viii
13 List of Tables 21 Summary of Redhill primary school results from JSP dataset 6 22 Parameter estimates for model including Sex and NonManual covariates for Redhill primary school 8 23 Summaryofschools in the borough of Blackbridge 8 24 Parameter estimates for ANOVA and ANCOVA models for the boroughofblackbridge dataset Parameter estimates for two variance components models using both IGLS and RIGLS for Borough of Blackbridge dataset Parameter estimates for two variance components models using both IGLS and RIGLS for all schools in the JSP dataset Comparison between tted values using the ANOVA model and the variance components model 1 using RIGLS Parameter estimates for random slopes model using both IGLS and RIGLS for all schools in the JSP dataset Comparison between tted regression lines produced by separate regressions and the random slopes model Parameter estimates for the two logistic regression models tted to the Blackbridge dataset Parameter estimates for the twolevel logistic regression models tted to the JSP dataset Comparison between MCMC methods for tting a bivariate normal model with unknown mean vector Comparison between 95% condence intervals and Bayesian credible intervals in bivariate normal model 51 ix
14 41 Summary of study designs for variance components model simulation Summary of times for Gibbs sampling in the variance components model with dierent study designs for 50,000 iterations Summary of Raftery Lewis convergence times (thousands of iterations) for various studies Summary of simulation lengths for Gibbs sampling the variance components model with dierent study designs Estimates of relative bias for the variance parameters using dierent methods and dierent studies True level 2/1 variance values are10and Estimates of relative bias for the variance parameters using dierent methods and dierent true values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the xed eect parameter using dierent methods and dierent studies True values for the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the xed eect parameter using dierent studies True values for the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the xed eect parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the xed eect parameter using dierent true parameter values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 2 variance parameter using dierent methods and dierent studies True values of the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates 73 x
15 412 Average 90%/95% interval widths for the level 2 variance parameter using dierent studies True values of the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 2 variance parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 2 variance parameter using dierent true parameter values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 1 variance parameter using dierent methods and dierent studies True values of the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 1 variance parameter using dierent studies True values of the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 1 variance parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 1 variance parameter using dierent true parameter values All runs use study design Summary of results for the level 2 variance parameter, u 2 using the RIGLS method and inverse gamma intervals Summary of the convergence for the random slopes regression with the maximum likelihood based methods (IGLS/RIGLS) The study design is given in terms of the number of level 2 units and whether the study is balanced (B) or unbalanced (U) 86 xi
16 421 Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 0and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 1:4 and u11 =0:5 Only 982 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = ;1:4 and u11 =0:5 Only 984 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 0:5 and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = ;0:5 and u11 =0:5 Only 998 runs Summary of results for the random slopes regression with the 48 schools balanced design with parameter values, u00 = 5 u01 = 0:0 and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 12 schools unbalanced design with parameter values, u00 =5 u01 = 0:0 and u11 =0:5 Only 877 runs Summary of results for the random slopes regression with the 12 schools balanced design with parameter values, u00 = 5 u01 = 0:0 and u11 =0:5 Only 990 runs Optimal scale factors for proposal variances and best acceptance rates for several models Demonstration of Adaptive Method 1 for parameters 0 and 1 using arbitrary (1000) starting values Comparison of results for the random slopes regression model on the JSP dataset using uniform priors for the variances, and dierent MCMC methods Each method was run for 50,000 iterations after a burnin of xii
17 54 Demonstration of Adaptive Method 2 for parameters 0 and 1 using arbitrary (1000) starting values Demonstration of Adaptive Method 3 for the parameter vector using RIGLS starting values Comparison of results for the random slopes regression model on the JSP dataset using uniform priors for the variances, and dierent block updating MCMC methods Each method was run for 50,000 iterations after a burnin of Comparison of results from the quasilikelihood methods and the MCMC methods for the voting intention dataset The MCMC method is based on a run of 50,000 iterations after a burnin of 500 and adapting period Optimal scale factors for proposal variances and best acceptance rates for the voting intentions model Summary of results (with Monte Carlo standard errors) for the rst 25 datasets of the Rodriguez Goldman example Summary of results (with Monte Carlo standard errors) for the Rodriguez Goldman example with 500 generated datasets Comparison between three MCMC methods for a univariate normal model with unknown variance Comparison between two MCMC methods for a bivariate normal model with unknown variance matrix Comparison between IGLS/RIGLS and MCMC method on a simulated dataset with the layout of the JSP dataset Comparison between RIGLS and MCMC method 2 on three models with complex variation tted to the JSP dataset 177 xiii
18 Chapter 1 Introduction 11 Objectives Multilevel modelling has recently become an increasingly interesting and applicable statistical tool Many areas of application t readily into a multilevel structure Goldstein and Spiegelhalter (1996) illustrate the use of multilevel modelling in two leading application areas, health and education other application areas include household surveys and animal growth studies Several packages have been written to t multilevel models MLn (Rasbash and Woodhouse (1995)), HLM (Bryk et al (1988), Bryk and Raudenbush (1992)), and VARCL (Longford (1987), Longford (1988)) are all packages which use as their tting mechanisms, maximum likelihood or empirical Bayes methodology These methods are used to nd estimates of parameters of interest in complicated models where exact methods would involve intractable integrations Another technique that has come to the forefront of statistical research over the last decade or so is the use of Markov chain Monte Carlo (MCMC) simulation methods (Gelfand and Smith 1990) With the increase in computer power, both in speed of computation and in memory capacity, techniques that were theoretical ideas thirty years ago are now practical reality The structure of the multilevel model with its interdependence between variables makes it an ideal area of application for MCMC techniques Draper (1995) describes the use of multilevel modelling in the social sciences and recommends greater use of MCMC methods in this eld 1
19 When MCMC methods were rst introduced, if statisticians wanted to t a complicated model they would program up their own MCMC sampler for the problem they were considering and use it to solve that problem More recently a general purpose MCMC sampler, BUGS (Spiegelhalter et al 1995) has been produced that will t a wide range of models in many application areas BUGS uses a technique called Gibbs sampling to t its models using an adaptive rejection algorithm described in Gilks and Wild (1992) In this thesis I am interested in studying multilevel models, and comparing the maximum likelihood based methods in the package MLn with MCMC methods I will parallel the work of BUGS and consider tting various families of multilevel models using both Gibbs sampling and MetropolisHastings sampling methods I will also consider how the maximum likelihood methods can be used to give the MCMC methods good starting values and suitable proposal distributions for MetropolisHastings sampling The package MLwiN (Goldstein et al 1998) is the new version of MLn and some of its new features are a result of the work contained in this thesis MLwiN contains for the rst time MCMC methodology as well as the existing maximum likelihood based methods 12 Summary of Thesis In the next chapter I will discuss some of the background to multilevel modelling using as an example an educational dataset Iwillintroduce multilevel modelling as an extension to linear modelling and explain briey how the existing maximum likelihood methods in MLn t multilevel models In Chapter 3 I will consider MCMC simulation techniques and summarise the main techniques, Metropolis sampling, Gibbs sampling and Hastings sampling I will explain how such techniques are used and how to get estimates from the chains they produce I will also consider convergence issues when using Markov chains and motivate all the methods with a simple example In Chapter 4 I will consider two very simple multilevel models, the twolevel variance components model, and the random slopes regression model, both introduced in Chapter 2 I will use these models to illustrate the important issue of choosing general `diuse' prior distributions when using MCMC methods The 2
20 chapter will consist of two large simulation experiments to compare and contrast the IGLS and RIGLS maximum likelihood methods with MCMC methods using various prior distributions under dierent scenarios In Chapter 5 I will discuss some more general algorithms that will t N level Gaussian multilevel models I will give three algorithms, rstly Gibbs sampling and then two hybrid Gibbs Metropolis samplers: the rst containing univariate updating steps, and the second block updating steps For each hybrid sampler I will also describe an adaptive Metropolis technique to improve itsmixing I will then compare all the samplers through some simple examples In Chapter 6 I will discuss multilevel logistic regression models I will consider one of the hybrid samplers introduced in the previous chapter and show howitcan be modied to t these new models These models are a family that maximum likelihood techniques perform particularly badly on I will therefore compare the maximum likelihood based methods with the new hybrid sampler via another simulation experiment In Chapter 7 I will introduce a complex variation structure at level 1 as a generalisation of the Gaussian models introduced in Chapter 5 I will then implement two Hastings updating techniques for the level 1 variance parameters that aim to sample from such models Firstly a technique based on an inverse Wishart proposal distribution and secondly a technique based on a truncated normal proposal distribution I will then compare the results of both methods to the maximum likelihood methods In Chapter 8 I will discuss other multilevel models that have not been tted in the previous chapters and add some general conclusions about the thesis as awhole 3
21 Chapter 2 Multi Level Models and MLn 21 Introduction In the introduction I mentioned several applications that contain datasets where a multilevel structure is appropriate The package MLn (Rasbash and Woodhouse 1995) was written at the Institute of Education primarily to t models in the area of education although it can be used in many of the other applications of multilevel modelling In this chapter I intend to consider, through examples, some statistical problems that arise in the eld of education These problems will increase in complexity to incorporate multilevel modelling I will explain how the maximum likelihood methods in MLn can be used to t the models as each new model is introduced The dataset used in this chapter is the Junior School Project (JSP) dataset analysed in Woodhouse et al (1995) 211 JSP dataset The JSP is a longitudinal study of approximately 2000 pupils who entered junior school in 1980 Woodhouse et al (1995) analyse a subset of the data containing 887 pupils from 48 primary schools taken from the Inner London Education Authority (ILEA) For each child they consider his/her Maths scores in two tests, marked out of 40, taken in years 3 and 5, along with other variables that measure the child's background I will consider smaller subsets of this subset in the models considered in this chapter Any names used in the examples are ctitious, and are simply used to aid my descriptions 4
22 I will now consider as my rst dataset the sample of pupils from one school participating in the Junior School project, and consider how to statistically describe information on an individual pupil 22 Analysing Redhill school data Redhill Primary school is the 5th school in the JSP dataset and the sample of pupils participating in the JSP has 25 pupils who sat Maths tests in years 3 and 5 I will denote the Maths scores, out of a possible 40, in years 3 and 5 as M3 and M5 respectively When considering the data from one school, it is the individual pupils' marks that are of interest The data for Redhill school are given in Table 21 The individual pupils and their parents will be interested in how they, or their children are doing in terms of what marks were achieved, and how these marks compare with the other pupils in the school Consider John Smith, pupil 10, who achieved 30 in both his M3 andm5 test scores, or equivalently 75% in each test If this is the only information available then John Smith appears to have made steady progress in mathematics If instead the marks for the whole class are available then each child could be given a ranking to indicate where he/she nished in the class It can now be seen that John Smith ranked equal eighth in the third year test but only equal eighteenth in the second test, so although he got the same mark in each test, compared to the rest of the class he has done worse in the second test This is because although his marks have stayed constant the mean mark for the class has risen from 278 to 32 This may be because the second test is in fact comparatively easier than the rst test, or the teaching between the two tests has improved the children's average performance With only the data given it is impossible to distinguish between these two reasons for the improved average mark 221 Linear regression A better way to compare John Smith's two marks is to perform a regression of the M5 marks on the M3 marks This will be the rst model to be tted to the 5
23 Table 21: Summary of Redhill primary school results from JSP dataset Pupil M3 Rank M5 Rank M/NM Sex M M M M NM F NM M M F NM F NM M NM F NM F M M NM F NM M M F M F NM M NM M M F M F NM F NM F NM F NM M M M NM M NM M dataset and can be written as : M5 i = M3 i + e i where the e i are the residuals and are assumed to be distributed normally with mean zero and variance 2 The linear regression problem is studied in great detail in basic statistics courses and the least squares estimates are as follows : ^ 0 = y ; ^ 1 x 6
EVALUATION OF GAUSSIAN PROCESSES AND OTHER METHODS FOR NONLINEAR REGRESSION. Carl Edward Rasmussen
EVALUATION OF GAUSSIAN PROCESSES AND OTHER METHODS FOR NONLINEAR REGRESSION Carl Edward Rasmussen A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Graduate
More informationSome statistical heresies
The Statistician (1999) 48, Part 1, pp. 1±40 Some statistical heresies J. K. Lindsey Limburgs Universitair Centrum, Diepenbeek, Belgium [Read before The Royal Statistical Society on Wednesday, July 15th,
More informationBootstrap Methods and Their Application
Bootstrap Methods and Their Application cac Davison and DV Hinkley Contents Preface i 1 Introduction 1 2 The Basic Bootstraps 11 21 Introduction 11 22 Parametric Simulation 15 23 Nonparametric Simulation
More informationFlexible and efficient Gaussian process models for machine learning
Flexible and efficient Gaussian process models for machine learning Edward Lloyd Snelson M.A., M.Sci., Physics, University of Cambridge, UK (2001) Gatsby Computational Neuroscience Unit University College
More informationAN INTRODUCTION TO MARKOV CHAIN MONTE CARLO METHODS AND THEIR ACTUARIAL APPLICATIONS. Department of Mathematics and Statistics University of Calgary
AN INTRODUCTION TO MARKOV CHAIN MONTE CARLO METHODS AND THEIR ACTUARIAL APPLICATIONS DAVID P. M. SCOLLNIK Department of Mathematics and Statistics University of Calgary Abstract This paper introduces the
More informationWe show that social scientists often do not take full advantage of
Making the Most of Statistical Analyses: Improving Interpretation and Presentation Gary King Michael Tomz Jason Wittenberg Harvard University Harvard University Harvard University Social scientists rarely
More informationWhat LargeScale, Survey Research Tells Us About Teacher Effects On Student Achievement: Insights from the Prospects Study of Elementary Schools
What LargeScale, Survey Research Tells Us About Teacher Effects On Student Achievement: Insights from the Prospects Study of Elementary Schools Brian Rowan, Richard Correnti, and Robert J. Miller CPRE
More informationTHE PROBLEM OF finding localized energy solutions
600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Reweighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,
More informationIntroduction to Linear Regression
14. Regression A. Introduction to Simple Linear Regression B. Partitioning Sums of Squares C. Standard Error of the Estimate D. Inferential Statistics for b and r E. Influential Observations F. Regression
More informationAn Introduction to MCMC for Machine Learning
Machine Learning, 50, 5 43, 2003 c 2003 Kluwer Academic Publishers. Manufactured in The Netherlands. An Introduction to MCMC for Machine Learning CHRISTOPHE ANDRIEU C.Andrieu@bristol.ac.uk Department of
More informationThe result of 11+ Selection: An Investigation into Opportunities and Outcomes for Pupils in Selective LEAs
THE CENTRE FOR MARKET AND PUBLIC ORGANISATION The Centre for Market and Public Organisation (CMPO) is a leading research centre, combining expertise in economics, geography and law. Our objective is to
More informationBootstrap con"dence intervals: when, which, what? A practical guide for medical statisticians
STATISTICS IN MEDICINE Statist. Med. 2000; 19:1141}1164 Bootstrap con"dence intervals: when, which, what? A practical guide for medical statisticians James Carpenter* and John Bithell Medical Statistics
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More informationAn Introduction to Regression Analysis
The Inaugural Coase Lecture An Introduction to Regression Analysis Alan O. Sykes * Regression analysis is a statistical tool for the investigation of relationships between variables. Usually, the investigator
More informationThe InStat guide to choosing and interpreting statistical tests
Version 3.0 The InStat guide to choosing and interpreting statistical tests Harvey Motulsky 19902003, GraphPad Software, Inc. All rights reserved. Program design, manual and help screens: Programming:
More informationEfficient Algorithms for Sorting and Synchronization Andrew Tridgell
Efficient Algorithms for Sorting and Synchronization Andrew Tridgell A thesis submitted for the degree of Doctor of Philosophy at The Australian National University February 1999 Except where otherwise
More informationOn the Optimality of the Simple Bayesian. given the class, but the question of whether other sucient conditions for its optimality exist has
,, 1{30 () c Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. On the Optimality of the Simple Bayesian Classier under ZeroOne Loss PEDRO DOMINGOS pedrod@ics.uci.edu MICHAEL PAZZANI
More informationScalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights
Seventh IEEE International Conference on Data Mining Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights Robert M. Bell and Yehuda Koren AT&T Labs Research 180 Park
More informationBIS RESEARCH PAPER NO. 112 THE IMPACT OF UNIVERSITY DEGREES ON THE LIFECYCLE OF EARNINGS: SOME FURTHER ANALYSIS
BIS RESEARCH PAPER NO. 112 THE IMPACT OF UNIVERSITY DEGREES ON THE LIFECYCLE OF EARNINGS: SOME FURTHER ANALYSIS AUGUST 2013 1 THE IMPACT OF UNIVERSITY DEGREES ON THE LIFECYCLE OF EARNINGS: SOME FURTHER
More informationThe Capital Asset Pricing Model: Some Empirical Tests
The Capital Asset Pricing Model: Some Empirical Tests Fischer Black* Deceased Michael C. Jensen Harvard Business School MJensen@hbs.edu and Myron Scholes Stanford University  Graduate School of Business
More informationMisunderstandings between experimentalists and observationalists about causal inference
J. R. Statist. Soc. A (2008) 171, Part 2, pp. 481 502 Misunderstandings between experimentalists and observationalists about causal inference Kosuke Imai, Princeton University, USA Gary King Harvard University,
More informationA Googlelike Model of Road Network Dynamics and its Application to Regulation and Control
A Googlelike Model of Road Network Dynamics and its Application to Regulation and Control Emanuele Crisostomi, Steve Kirkland, Robert Shorten August, 2010 Abstract Inspired by the ability of Markov chains
More informationSampling 50 Years After Shannon
Sampling 50 Years After Shannon MICHAEL UNSER, FELLOW, IEEE This paper presents an account of the current state of sampling, 50 years after Shannon s formulation of the sampling theorem. The emphasis is
More informationRegression. Chapter 2. 2.1 Weightspace View
Chapter Regression Supervised learning can be divided into regression and classification problems. Whereas the outputs for classification are discrete class labels, regression is concerned with the prediction
More informationAn Introduction to Variable and Feature Selection
Journal of Machine Learning Research 3 (23) 11571182 Submitted 11/2; Published 3/3 An Introduction to Variable and Feature Selection Isabelle Guyon Clopinet 955 Creston Road Berkeley, CA 9478151, USA
More informationMaking Regression Analysis More Useful, II: Dummies and Trends
7 Making Regression Analysis More Useful, II: Dummies and Trends LEARNING OBJECTIVES j Know what a dummy variable is and be able to construct and use one j Know what a trend variable is and be able to
More informationTheir Scales with a ScaleSpace Primal Sketch: A Method for FocusofAttention. Tony Lindeberg
Detecting Salient BlobLike Image Structures and Their Scales with a ScaleSpace Primal Sketch: A Method for FocusofAttention Tony Lindeberg Computational Vision and Active Perception Laboratory (CVAP)
More informationGenerative or Discriminative? Getting the Best of Both Worlds
BAYESIAN STATISTICS 8, pp. 3 24. J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West (Eds.) c Oxford University Press, 2007 Generative or Discriminative?
More informationE. F. Allan, S. Abeyasekera & R. D. Stern
E. F. Allan, S. Abeyasekera & R. D. Stern January 2006 The University of Reading Statistical Services Centre Guidance prepared for the DFID Forestry Research Programme Foreword The production of this
More informationA point process model for the dynamics of limit order books
A point process model for the dynamics of limit order books Ekaterina Vinkovskaya Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts
More information