Size: px
Start display at page:

Download ""

Transcription

1 Applying MCMC Methods to Multi-level Models submitted by William J Browne for the degree of PhD of the University of Bath 1998 COPYRIGHT Attention is drawn tothefactthatcopyright of this thesis rests with its author This copy of the thesis has been supplied on the condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the prior written consent of the author This thesis may bemadeavailable for consultation within the University Library and may be photocopied or lent to other libraries for the purposes of consultation Signature of Author William J Browne

2 To Health, Happiness and Honesty

3 Summary Multi-level modelling and Markov chain Monte Carlo methods are two areas of statistics that have become increasingly popular recently due to improvements in computer capabilities, both in storage and speed of operation The aim of this thesis is to combine the two areas by tting multi-level models using Markov chain Monte Carlo (MCMC) methods This task has been split into three parts in this thesis Firstly the types of problems that are tted in multilevel modelling are identied and the existing maximum likelihood methods are investigated Secondly MCMC algorithms for these models are derived and nally these methods are compared to the maximum likelihood based methods both in terms of estimate bias and interval coverage properties Three main groups of multi-level models are considered Firstly N level Gaussian models, secondly binary response multi-level logistic regression models and nally Gaussian models with complex variation at level 1 Two simple 2 level Gaussian models are rstly considered and it is shown how to t these models using the Gibbs sampler Then extensive simulation studies are carried out to compare the Gibbs sampler method with maximum likelihood methods on these two models For the general N level Gaussian models, algorithms for the Gibbs sampler and two alternative hybrid Metropolis Gibbs methods are given and these three methods are then compared with each other One of the hybrid Metropolis Gibbs methods is adapted to t binary response multi-level models This method is then compared with two quasi-likelihood methods via a simulation study on one binary response model where the quasilikelihood methods perform particularly badly All of the above models can also be tted using the Gibbs sampling method using the adaptive rejection algorithm in the BUGS package (Spiegelhalter et al 1994) Finally Gaussian models with complex variation at level 1 which cannot be tted in BUGS are considered Two methods based on Hastings update steps are given and are tested on some simple examples The MCMC methods in this thesis have been added to the multi-level modelling package MLwiN (Goldstein et al 1998) as a by-product of this research

4 Acknowledgements I would rstly like to thank my supervisor, Dr David Draper whose research in the elds of hierarchical modelling and Bayesian statistics motivated this PhD I would also like to thank him for his advice and assistance throughout both my MSc and PhD I would like to thank my parents for supporting me both nancially and emotionally through my rst degree and beyond I would like to thank the multilevel models project team at the Institute of Education, in particular Jon Rasbash and Professor Harvey Goldstein for allowing me to work with them on the MLwiN package I would also like to thank them for their advice and assistance while I have been working on the package I would like to thank my brother Edward and his ance Meriel for arranging their wedding a month before I am scheduled to nish this thesis This way I can spread my worries between my PhD andmy best man's speech Iwould like to thank my girlfriends over the last three years for helping me through various parts of my PhD Thanks for giving me love and support when I needed it and making my life both happy and interesting I would like to thank the other members of the statistics group at Bath for teaching me all I know about statistics today I would like to thank my fellow oce mates, past and present for their humour, conversation and friendship and for joining me in my many pointless conversations Thanks to family and friends both in Bath and elsewhere Special thanks are due to the EPSRC for their nancial support \The only thing I know is that I don't know anything" Socrates

5 Contents 1 Introduction 1 11 Objectives 1 12 Summary of Thesis 2 2 Multi Level Models and MLn 4 21 Introduction JSP dataset 4 22 Analysing Redhill school data Linear regression Linear models 7 23 Analysing data on the four schools in the borough of Blackbridge ANOVA model ANCOVA model Combined regression Two level modelling Iterative generalised least squares Restricted iterative generalised least squares Fitting variance components models to the Blackbridge dataset Fitting variance components models to the JSP dataset Random slopes model Fitting models to pass/fail data Extending to multi-level modelling Summary 21 i

6 3 Markov Chain Monte Carlo Methods Background Bayesian inference Metropolis sampling Proposal distributions Metropolis-Hastings sampling Gibbs sampling Rejection sampling Adaptive rejection sampling Gibbs sampler as a special case of the Metropolis-Hastings algorithm Data summaries Measures of location Measures of spread Plots Convergence issues Length of burn-in Mixing properties of Markov chains Multi-modal models Summary Use of MCMC methods in multi-level modelling Example - Bivariate normal distribution Metropolis sampling Metropolis-Hasting sampling Gibbs sampling Results Summary 51 4 Gaussian Models 1 - Introduction Introduction Prior distributions Informative priors Non-informative priors Priors for xed eects 55 ii

7 424 Priors for single variances Priors for variance matrices Level variance components model Gibbs sampling algorithm Simulation method Results : Bias Results : Coverage probabilities and interval widths Improving maximum likelihood method interval estimates for u Summary of results Random slopes regression model Gibbs sampling algorithm Simulation method Results Conclusions Simulation results Priors in MLwiN Gaussian Models 2 - General Models General N level Gaussian hierarchical linear models Gibbs sampling approach Generalising to N levels Algorithm Computational considerations Method 2 : Metropolis Gibbs hybrid method with univariate updates Algorithm Choosing proposal distribution variances Adaptive Metropolis univariate normal proposals Method 3:Metropolis Gibbs hybrid method with block updates Algorithm Choosing proposal distribution variances Adaptive multivariate normal proposal distributions Summary Timing considerations 138 iii

8 6 Logistic Regression Models Introduction Multi-level binary response logistic regression models Metropolis Gibbs hybrid method with univariate updates Other existing methods Example 1 : Voting intentions dataset Background Model Results Substantive Conclusions Optimum proposal distributions Example 2 : Guatemalan child health dataset Background Model Original 25 datasets Simulating more datasets Conclusions Summary Gaussian Models 3 - Complex Variation at level Model denition Updating methods for a scalar variance Metropolis algorithm for log Hastings algorithm for Example : Normal observations with an unknown variance Results Updating methods for a variance matrix Hastings algorithm with an inverse Wishart proposal Example : Bivariate normal observations with an unknown variance matrix Results Applying inverse Wishart updates to complex variation at level MCMC algorithm Example iv

9 743 Conclusions Method 2: Using truncated normal Hastings update steps Update steps at level 1 for JSP example Proposal distributions Example 2 : Non-positive denite and incomplete variance matrices at level General algorithm for truncated normal proposal method Summary Conclusions and Further Work Conclusions MCMC options in the MLwiN package Further work Binomial responses Multinomial models Poisson responses for count data Extensions to complex variation at level Multivariate response models 188 v

10 List of Figures 2-1 Plot of the regression lines for the four schools in the Borough of Blackbridge Tree diagram for the Borough of Blackbridge Histogram of 1 using the Gibbs sampling method Kernel density plot of 1 using the Gibbs sampling method and a Gaussian kernel with a large value of the window width h Traces of parameter 1 and the running mean of 1 for a Metropolis run that converges after about 50 iterations Upper solid line in lower panel is running mean with rst 50 iterations discarded ACF and PACF for parameter 1 for a Gibbs sampling run of length 5000 that is mixing well and a Metropolis run that is not mixing very well Kernel density plot of 2 using the Gibbs sampling method and a Gaussian kernel Plots of the Raftery Lewis ^N values for various values of p, the proposal distribution standard deviation Plot of the MCMC diagnostic window in the package MLwiN for the parameter 1 from a random slopes regression model Plot of normal prior distributions over the range ({5,5) with mean 0andvariances 1,2,5,10 and 50 respectively Plots of biases obtained for the various methods against study design and parameter settings Trajectories plot of IGLS estimates for run of random slopes regression model where convergence is not achieved 85 vi

11 4-4 Plots of biases obtained for the various methods tting the random slopes regression model against value of u01 (Fixed eects parameters and level 1 variance parameter) Plots of biases obtained for the various methods tting the random slopes regression model against value of u01 (Level 2 variance parameters) Plots of biases obtained for the various methods tting the random slopes regression model against study design (Fixed eects parameters and level 1 variance parameter) Plots of biases obtained for the various methods tting the random slopes regression model against study design (Level 2 variance parameters) Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the variance components model on the JSP dataset Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the proposal variance and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 1 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the multivariate normal proposal distribution and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 0 parameter in the random slopes regression model on the JSP dataset Plots of the eect of varying the scale factor for the multivariate normal proposal distribution and hence the Metropolis acceptance rate on the Raftery Lewis diagnostic for the 1 parameter in the random slopes regression model on the JSP dataset 131 vii

12 6-1 Plot of the eect of varying the scale factor for the univariate Normal proposal distribution rate on the Raftery Lewis diagnostic for the u 2 parameter in the voting intentions dataset Plots comparing the actual coverage of the four estimation methods with their nominal coverage for the parameters 0 1 and Plots comparing the actual coverage of the four estimation methods with their nominal coverage for the parameters 3 v 2 and u Plots of truncated univariate normal proposal distributions for a parameter, A is the current value, c and B is the proposed new value, M is max and m is min, the truncation points The distributions in (i) and (iii) have mean c, while the distributions in (ii) and (iv) have mean 175 viii

13 List of Tables 21 Summary of Redhill primary school results from JSP dataset 6 22 Parameter estimates for model including Sex and Non-Manual covariates for Redhill primary school 8 23 Summaryofschools in the borough of Blackbridge 8 24 Parameter estimates for ANOVA and ANCOVA models for the boroughofblackbridge dataset Parameter estimates for two variance components models using both IGLS and RIGLS for Borough of Blackbridge dataset Parameter estimates for two variance components models using both IGLS and RIGLS for all schools in the JSP dataset Comparison between tted values using the ANOVA model and the variance components model 1 using RIGLS Parameter estimates for random slopes model using both IGLS and RIGLS for all schools in the JSP dataset Comparison between tted regression lines produced by separate regressions and the random slopes model Parameter estimates for the two logistic regression models tted to the Blackbridge dataset Parameter estimates for the two-level logistic regression models tted to the JSP dataset Comparison between MCMC methods for tting a bivariate normal model with unknown mean vector Comparison between 95% condence intervals and Bayesian credible intervals in bivariate normal model 51 ix

14 41 Summary of study designs for variance components model simulation Summary of times for Gibbs sampling in the variance components model with dierent study designs for 50,000 iterations Summary of Raftery Lewis convergence times (thousands of iterations) for various studies Summary of simulation lengths for Gibbs sampling the variance components model with dierent study designs Estimates of relative bias for the variance parameters using dierent methods and dierent studies True level 2/1 variance values are10and Estimates of relative bias for the variance parameters using dierent methods and dierent true values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the xed eect parameter using dierent methods and dierent studies True values for the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the xed eect parameter using dierent studies True values for the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the xed eect parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the xed eect parameter using dierent true parameter values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 2 variance parameter using dierent methods and dierent studies True values of the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates 73 x

15 412 Average 90%/95% interval widths for the level 2 variance parameter using dierent studies True values of the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 2 variance parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 2 variance parameter using dierent true parameter values All runs use study design Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 1 variance parameter using dierent methods and dierent studies True values of the variance parameters are 10 and 40 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 1 variance parameter using dierent studies True values of the variance parameters are 10 and Comparison of actual coverage percentage values for nominal 90% and 95% intervals for the level 1 variance parameter using dierent methods and dierent true values All runs use study design 7 Approximate MCSEs are 028%/015% for 90%/95% coverage estimates Average 90%/95% interval widths for the level 1 variance parameter using dierent true parameter values All runs use study design Summary of results for the level 2 variance parameter, u 2 using the RIGLS method and inverse gamma intervals Summary of the convergence for the random slopes regression with the maximum likelihood based methods (IGLS/RIGLS) The study design is given in terms of the number of level 2 units and whether the study is balanced (B) or unbalanced (U) 86 xi

16 421 Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 0and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 1:4 and u11 =0:5 Only 982 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = ;1:4 and u11 =0:5 Only 984 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = 0:5 and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 48 schools unbalanced design with parameter values, u00 =5 u01 = ;0:5 and u11 =0:5 Only 998 runs Summary of results for the random slopes regression with the 48 schools balanced design with parameter values, u00 = 5 u01 = 0:0 and u11 =0:5 All 1000 runs Summary of results for the random slopes regression with the 12 schools unbalanced design with parameter values, u00 =5 u01 = 0:0 and u11 =0:5 Only 877 runs Summary of results for the random slopes regression with the 12 schools balanced design with parameter values, u00 = 5 u01 = 0:0 and u11 =0:5 Only 990 runs Optimal scale factors for proposal variances and best acceptance rates for several models Demonstration of Adaptive Method 1 for parameters 0 and 1 using arbitrary (1000) starting values Comparison of results for the random slopes regression model on the JSP dataset using uniform priors for the variances, and dierent MCMC methods Each method was run for 50,000 iterations after a burn-in of xii

17 54 Demonstration of Adaptive Method 2 for parameters 0 and 1 using arbitrary (1000) starting values Demonstration of Adaptive Method 3 for the parameter vector using RIGLS starting values Comparison of results for the random slopes regression model on the JSP dataset using uniform priors for the variances, and dierent block updating MCMC methods Each method was run for 50,000 iterations after a burn-in of Comparison of results from the quasi-likelihood methods and the MCMC methods for the voting intention dataset The MCMC method is based on a run of 50,000 iterations after a burn-in of 500 and adapting period Optimal scale factors for proposal variances and best acceptance rates for the voting intentions model Summary of results (with Monte Carlo standard errors) for the rst 25 datasets of the Rodriguez Goldman example Summary of results (with Monte Carlo standard errors) for the Rodriguez Goldman example with 500 generated datasets Comparison between three MCMC methods for a univariate normal model with unknown variance Comparison between two MCMC methods for a bivariate normal model with unknown variance matrix Comparison between IGLS/RIGLS and MCMC method on a simulated dataset with the layout of the JSP dataset Comparison between RIGLS and MCMC method 2 on three models with complex variation tted to the JSP dataset 177 xiii

18 Chapter 1 Introduction 11 Objectives Multi-level modelling has recently become an increasingly interesting and applicable statistical tool Many areas of application t readily into a multilevel structure Goldstein and Spiegelhalter (1996) illustrate the use of multilevel modelling in two leading application areas, health and education other application areas include household surveys and animal growth studies Several packages have been written to t multi-level models MLn (Rasbash and Woodhouse (1995)), HLM (Bryk et al (1988), Bryk and Raudenbush (1992)), and VARCL (Longford (1987), Longford (1988)) are all packages which use as their tting mechanisms, maximum likelihood or empirical Bayes methodology These methods are used to nd estimates of parameters of interest in complicated models where exact methods would involve intractable integrations Another technique that has come to the forefront of statistical research over the last decade or so is the use of Markov chain Monte Carlo (MCMC) simulation methods (Gelfand and Smith 1990) With the increase in computer power, both in speed of computation and in memory capacity, techniques that were theoretical ideas thirty years ago are now practical reality The structure of the multilevel model with its interdependence between variables makes it an ideal area of application for MCMC techniques Draper (1995) describes the use of multi-level modelling in the social sciences and recommends greater use of MCMC methods in this eld 1

19 When MCMC methods were rst introduced, if statisticians wanted to t a complicated model they would program up their own MCMC sampler for the problem they were considering and use it to solve that problem More recently a general purpose MCMC sampler, BUGS (Spiegelhalter et al 1995) has been produced that will t a wide range of models in many application areas BUGS uses a technique called Gibbs sampling to t its models using an adaptive rejection algorithm described in Gilks and Wild (1992) In this thesis I am interested in studying multi-level models, and comparing the maximum likelihood based methods in the package MLn with MCMC methods I will parallel the work of BUGS and consider tting various families of multi-level models using both Gibbs sampling and Metropolis-Hastings sampling methods I will also consider how the maximum likelihood methods can be used to give the MCMC methods good starting values and suitable proposal distributions for Metropolis-Hastings sampling The package MLwiN (Goldstein et al 1998) is the new version of MLn and some of its new features are a result of the work contained in this thesis MLwiN contains for the rst time MCMC methodology as well as the existing maximum likelihood based methods 12 Summary of Thesis In the next chapter I will discuss some of the background to multi-level modelling using as an example an educational dataset Iwillintroduce multi-level modelling as an extension to linear modelling and explain briey how the existing maximum likelihood methods in MLn t multi-level models In Chapter 3 I will consider MCMC simulation techniques and summarise the main techniques, Metropolis sampling, Gibbs sampling and Hastings sampling I will explain how such techniques are used and how to get estimates from the chains they produce I will also consider convergence issues when using Markov chains and motivate all the methods with a simple example In Chapter 4 I will consider two very simple multi-level models, the twolevel variance components model, and the random slopes regression model, both introduced in Chapter 2 I will use these models to illustrate the important issue of choosing general `diuse' prior distributions when using MCMC methods The 2

20 chapter will consist of two large simulation experiments to compare and contrast the IGLS and RIGLS maximum likelihood methods with MCMC methods using various prior distributions under dierent scenarios In Chapter 5 I will discuss some more general algorithms that will t N level Gaussian multi-level models I will give three algorithms, rstly Gibbs sampling and then two hybrid Gibbs Metropolis samplers: the rst containing univariate updating steps, and the second block updating steps For each hybrid sampler I will also describe an adaptive Metropolis technique to improve itsmixing I will then compare all the samplers through some simple examples In Chapter 6 I will discuss multi-level logistic regression models I will consider one of the hybrid samplers introduced in the previous chapter and show howitcan be modied to t these new models These models are a family that maximum likelihood techniques perform particularly badly on I will therefore compare the maximum likelihood based methods with the new hybrid sampler via another simulation experiment In Chapter 7 I will introduce a complex variation structure at level 1 as a generalisation of the Gaussian models introduced in Chapter 5 I will then implement two Hastings updating techniques for the level 1 variance parameters that aim to sample from such models Firstly a technique based on an inverse Wishart proposal distribution and secondly a technique based on a truncated normal proposal distribution I will then compare the results of both methods to the maximum likelihood methods In Chapter 8 I will discuss other multi-level models that have not been tted in the previous chapters and add some general conclusions about the thesis as awhole 3

21 Chapter 2 Multi Level Models and MLn 21 Introduction In the introduction I mentioned several applications that contain datasets where a multi-level structure is appropriate The package MLn (Rasbash and Woodhouse 1995) was written at the Institute of Education primarily to t models in the area of education although it can be used in many of the other applications of multi-level modelling In this chapter I intend to consider, through examples, some statistical problems that arise in the eld of education These problems will increase in complexity to incorporate multi-level modelling I will explain how the maximum likelihood methods in MLn can be used to t the models as each new model is introduced The dataset used in this chapter is the Junior School Project (JSP) dataset analysed in Woodhouse et al (1995) 211 JSP dataset The JSP is a longitudinal study of approximately 2000 pupils who entered junior school in 1980 Woodhouse et al (1995) analyse a subset of the data containing 887 pupils from 48 primary schools taken from the Inner London Education Authority (ILEA) For each child they consider his/her Maths scores in two tests, marked out of 40, taken in years 3 and 5, along with other variables that measure the child's background I will consider smaller subsets of this subset in the models considered in this chapter Any names used in the examples are ctitious, and are simply used to aid my descriptions 4

22 I will now consider as my rst dataset the sample of pupils from one school participating in the Junior School project, and consider how to statistically describe information on an individual pupil 22 Analysing Redhill school data Redhill Primary school is the 5th school in the JSP dataset and the sample of pupils participating in the JSP has 25 pupils who sat Maths tests in years 3 and 5 I will denote the Maths scores, out of a possible 40, in years 3 and 5 as M3 and M5 respectively When considering the data from one school, it is the individual pupils' marks that are of interest The data for Redhill school are given in Table 21 The individual pupils and their parents will be interested in how they, or their children are doing in terms of what marks were achieved, and how these marks compare with the other pupils in the school Consider John Smith, pupil 10, who achieved 30 in both his M3 andm5 test scores, or equivalently 75% in each test If this is the only information available then John Smith appears to have made steady progress in mathematics If instead the marks for the whole class are available then each child could be given a ranking to indicate where he/she nished in the class It can now be seen that John Smith ranked equal eighth in the third year test but only equal eighteenth in the second test, so although he got the same mark in each test, compared to the rest of the class he has done worse in the second test This is because although his marks have stayed constant the mean mark for the class has risen from 278 to 32 This may be because the second test is in fact comparatively easier than the rst test, or the teaching between the two tests has improved the children's average performance With only the data given it is impossible to distinguish between these two reasons for the improved average mark 221 Linear regression A better way to compare John Smith's two marks is to perform a regression of the M5 marks on the M3 marks This will be the rst model to be tted to the 5

23 Table 21: Summary of Redhill primary school results from JSP dataset Pupil M3 Rank M5 Rank M/NM Sex M M M M NM F NM M M F NM F NM M NM F NM F M M NM F NM M M F M F NM M NM M M F M F NM F NM F NM F NM M M M NM M NM M dataset and can be written as : M5 i = M3 i + e i where the e i are the residuals and are assumed to be distributed normally with mean zero and variance 2 The linear regression problem is studied in great detail in basic statistics courses and the least squares estimates are as follows : ^ 0 = y ; ^ 1 x 6

Handling attrition and non-response in longitudinal data

Handling attrition and non-response in longitudinal data Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

More information

Parallelization Strategies for Multicore Data Analysis

Parallelization Strategies for Multicore Data Analysis Parallelization Strategies for Multicore Data Analysis Wei-Chen Chen 1 Russell Zaretzki 2 1 University of Tennessee, Dept of EEB 2 University of Tennessee, Dept. Statistics, Operations, and Management

More information

Statistics Graduate Courses

Statistics Graduate Courses Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Bayesian Statistics: Indian Buffet Process

Bayesian Statistics: Indian Buffet Process Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note

More information

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

More information

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics.

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics. Course Catalog In order to be assured that all prerequisites are met, students must acquire a permission number from the education coordinator prior to enrolling in any Biostatistics course. Courses are

More information

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not. Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

More information

INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

More information

Centre for Central Banking Studies

Centre for Central Banking Studies Centre for Central Banking Studies Technical Handbook No. 4 Applied Bayesian econometrics for central bankers Andrew Blake and Haroon Mumtaz CCBS Technical Handbook No. 4 Applied Bayesian econometrics

More information

Statistical Analysis with Missing Data

Statistical Analysis with Missing Data Statistical Analysis with Missing Data Second Edition RODERICK J. A. LITTLE DONALD B. RUBIN WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents Preface PARTI OVERVIEW AND BASIC APPROACHES

More information

Introduction to Logistic Regression

Introduction to Logistic Regression OpenStax-CNX module: m42090 1 Introduction to Logistic Regression Dan Calderon This work is produced by OpenStax-CNX and licensed under the Creative Commons Attribution License 3.0 Abstract Gives introduction

More information

STATISTICA Formula Guide: Logistic Regression. Table of Contents

STATISTICA Formula Guide: Logistic Regression. Table of Contents : Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

More information

PREDICTIVE DISTRIBUTIONS OF OUTSTANDING LIABILITIES IN GENERAL INSURANCE

PREDICTIVE DISTRIBUTIONS OF OUTSTANDING LIABILITIES IN GENERAL INSURANCE PREDICTIVE DISTRIBUTIONS OF OUTSTANDING LIABILITIES IN GENERAL INSURANCE BY P.D. ENGLAND AND R.J. VERRALL ABSTRACT This paper extends the methods introduced in England & Verrall (00), and shows how predictive

More information

The Basic Two-Level Regression Model

The Basic Two-Level Regression Model 2 The Basic Two-Level Regression Model The multilevel regression model has become known in the research literature under a variety of names, such as random coefficient model (de Leeuw & Kreft, 1986; Longford,

More information

Alabama Department of Postsecondary Education

Alabama Department of Postsecondary Education Date Adopted 1998 Dates reviewed 2007, 2011, 2013 Dates revised 2004, 2008, 2011, 2013, 2015 Alabama Department of Postsecondary Education Representing Alabama s Public Two-Year College System Jefferson

More information

Lab 8: Introduction to WinBUGS

Lab 8: Introduction to WinBUGS 40.656 Lab 8 008 Lab 8: Introduction to WinBUGS Goals:. Introduce the concepts of Bayesian data analysis.. Learn the basic syntax of WinBUGS. 3. Learn the basics of using WinBUGS in a simple example. Next

More information

Revenue Management with Correlated Demand Forecasting

Revenue Management with Correlated Demand Forecasting Revenue Management with Correlated Demand Forecasting Catalina Stefanescu Victor DeMiguel Kristin Fridgeirsdottir Stefanos Zenios 1 Introduction Many airlines are struggling to survive in today's economy.

More information

Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com

Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com Bayesian Machine Learning (ML): Modeling And Inference in Big Data Zhuhua Cai Google Rice University caizhua@gmail.com 1 Syllabus Bayesian ML Concepts (Today) Bayesian ML on MapReduce (Next morning) Bayesian

More information

Marketing Mix Modelling and Big Data P. M Cain

Marketing Mix Modelling and Big Data P. M Cain 1) Introduction Marketing Mix Modelling and Big Data P. M Cain Big data is generally defined in terms of the volume and variety of structured and unstructured information. Whereas structured data is stored

More information

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012 Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive Amounts

More information

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives

More information

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data

More information

Data analysis in supersaturated designs

Data analysis in supersaturated designs Statistics & Probability Letters 59 (2002) 35 44 Data analysis in supersaturated designs Runze Li a;b;, Dennis K.J. Lin a;b a Department of Statistics, The Pennsylvania State University, University Park,

More information

SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

More information

BayesX - Software for Bayesian Inference in Structured Additive Regression

BayesX - Software for Bayesian Inference in Structured Additive Regression BayesX - Software for Bayesian Inference in Structured Additive Regression Thomas Kneib Faculty of Mathematics and Economics, University of Ulm Department of Statistics, Ludwig-Maximilians-University Munich

More information

CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS

CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS Examples: Regression And Path Analysis CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS Regression analysis with univariate or multivariate dependent variables is a standard procedure for modeling relationships

More information

Introduction to Longitudinal Data Analysis

Introduction to Longitudinal Data Analysis Introduction to Longitudinal Data Analysis Longitudinal Data Analysis Workshop Section 1 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section 1: Introduction

More information

Dealing with Missing Data

Dealing with Missing Data Res. Lett. Inf. Math. Sci. (2002) 3, 153-160 Available online at http://www.massey.ac.nz/~wwiims/research/letters/ Dealing with Missing Data Judi Scheffer I.I.M.S. Quad A, Massey University, P.O. Box 102904

More information

Analysis of Bayesian Dynamic Linear Models

Analysis of Bayesian Dynamic Linear Models Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main

More information

Introducing the Multilevel Model for Change

Introducing the Multilevel Model for Change Department of Psychology and Human Development Vanderbilt University GCM, 2010 1 Multilevel Modeling - A Brief Introduction 2 3 4 5 Introduction In this lecture, we introduce the multilevel model for change.

More information

Multilevel Modelling of medical data

Multilevel Modelling of medical data Statistics in Medicine(00). To appear. Multilevel Modelling of medical data By Harvey Goldstein William Browne And Jon Rasbash Institute of Education, University of London 1 Summary This tutorial presents

More information

Statistics in Retail Finance. Chapter 6: Behavioural models

Statistics in Retail Finance. Chapter 6: Behavioural models Statistics in Retail Finance 1 Overview > So far we have focussed mainly on application scorecards. In this chapter we shall look at behavioural models. We shall cover the following topics:- Behavioural

More information

Linear Classification. Volker Tresp Summer 2015

Linear Classification. Volker Tresp Summer 2015 Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

More information

Introduction to Data Analysis in Hierarchical Linear Models

Introduction to Data Analysis in Hierarchical Linear Models Introduction to Data Analysis in Hierarchical Linear Models April 20, 2007 Noah Shamosh & Frank Farach Social Sciences StatLab Yale University Scope & Prerequisites Strong applied emphasis Focus on HLM

More information

A Basic Introduction to Missing Data

A Basic Introduction to Missing Data John Fox Sociology 740 Winter 2014 Outline Why Missing Data Arise Why Missing Data Arise Global or unit non-response. In a survey, certain respondents may be unreachable or may refuse to participate. Item

More information

Tutorial on Markov Chain Monte Carlo

Tutorial on Markov Chain Monte Carlo Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,

More information

A User s Guide to MLwiN

A User s Guide to MLwiN A User s Guide to MLwiN Version 2.10 by Jon Rasbash, Fiona Steele, William J. Browne & Harvey Goldstein Centre for Multilevel Modelling, University of Bristol ii A User s Guide to MLwiN Copyright 2009

More information

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis Int. Journal of Math. Analysis, Vol. 5, 2011, no. 1, 1-13 Review of the Methods for Handling Missing Data in Longitudinal Data Analysis Michikazu Nakai and Weiming Ke Department of Mathematics and Statistics

More information

Learning outcomes. Knowledge and understanding. Competence and skills

Learning outcomes. Knowledge and understanding. Competence and skills Syllabus Master s Programme in Statistics and Data Mining 120 ECTS Credits Aim The rapid growth of databases provides scientists and business people with vast new resources. This programme meets the challenges

More information

Chapter 2: Systems of Linear Equations and Matrices:

Chapter 2: Systems of Linear Equations and Matrices: At the end of the lesson, you should be able to: Chapter 2: Systems of Linear Equations and Matrices: 2.1: Solutions of Linear Systems by the Echelon Method Define linear systems, unique solution, inconsistent,

More information

4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4

4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4 4. Simple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/4 Outline The simple linear model Least squares estimation Forecasting with regression Non-linear functional forms Regression

More information

Schools Value-added Information System Technical Manual

Schools Value-added Information System Technical Manual Schools Value-added Information System Technical Manual Quality Assurance & School-based Support Division Education Bureau 2015 Contents Unit 1 Overview... 1 Unit 2 The Concept of VA... 2 Unit 3 Control

More information

Multivariate Normal Distribution

Multivariate Normal Distribution Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

More information

Imputing Missing Data using SAS

Imputing Missing Data using SAS ABSTRACT Paper 3295-2015 Imputing Missing Data using SAS Christopher Yim, California Polytechnic State University, San Luis Obispo Missing data is an unfortunate reality of statistics. However, there are

More information

Markov Chain Monte Carlo Simulation Made Simple

Markov Chain Monte Carlo Simulation Made Simple Markov Chain Monte Carlo Simulation Made Simple Alastair Smith Department of Politics New York University April2,2003 1 Markov Chain Monte Carlo (MCMC) simualtion is a powerful technique to perform numerical

More information

LAGUARDIA COMMUNITY COLLEGE CITY UNIVERSITY OF NEW YORK DEPARTMENT OF MATHEMATICS, ENGINEERING, AND COMPUTER SCIENCE

LAGUARDIA COMMUNITY COLLEGE CITY UNIVERSITY OF NEW YORK DEPARTMENT OF MATHEMATICS, ENGINEERING, AND COMPUTER SCIENCE LAGUARDIA COMMUNITY COLLEGE CITY UNIVERSITY OF NEW YORK DEPARTMENT OF MATHEMATICS, ENGINEERING, AND COMPUTER SCIENCE MAT 119 STATISTICS AND ELEMENTARY ALGEBRA 5 Lecture Hours, 2 Lab Hours, 3 Credits Pre-

More information

Introduction to Regression and Data Analysis

Introduction to Regression and Data Analysis Statlab Workshop Introduction to Regression and Data Analysis with Dan Campbell and Sherlock Campbell October 28, 2008 I. The basics A. Types of variables Your variables may take several forms, and it

More information

data visualization and regression

data visualization and regression data visualization and regression Sepal.Length 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 I. setosa I. versicolor I. virginica I. setosa I. versicolor I. virginica Species Species

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Fairfield Public Schools

Fairfield Public Schools Mathematics Fairfield Public Schools AP Statistics AP Statistics BOE Approved 04/08/2014 1 AP STATISTICS Critical Areas of Focus AP Statistics is a rigorous course that offers advanced students an opportunity

More information

Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach

Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Refik Soyer * Department of Management Science The George Washington University M. Murat Tarimcilar Department of Management Science

More information

NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( ) Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

More information

STAT3016 Introduction to Bayesian Data Analysis

STAT3016 Introduction to Bayesian Data Analysis STAT3016 Introduction to Bayesian Data Analysis Course Description The Bayesian approach to statistics assigns probability distributions to both the data and unknown parameters in the problem. This way,

More information

Tilburg University. Publication date: 1998. Link to publication

Tilburg University. Publication date: 1998. Link to publication Tilburg University Time Series Analysis of Non-Gaussian Observations Based on State Space Models from Both Classical and Bayesian Perspectives Durbin, J.; Koopman, S.J.M. Publication date: 1998 Link to

More information

International College of Economics and Finance Syllabus Probability Theory and Introductory Statistics

International College of Economics and Finance Syllabus Probability Theory and Introductory Statistics International College of Economics and Finance Syllabus Probability Theory and Introductory Statistics Lecturer: Mikhail Zhitlukhin. 1. Course description Probability Theory and Introductory Statistics

More information

THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE

THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE Batsirai Winmore Mazviona 1 Tafadzwa Chiduza 2 ABSTRACT In general insurance, companies need to use data on claims gathered from

More information

An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics

An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Slide 1 An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Dr. Christian Asseburg Centre for Health Economics Part 1 Slide 2 Talk overview Foundations of Bayesian statistics

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

Electronic Theses and Dissertations UC Riverside

Electronic Theses and Dissertations UC Riverside Electronic Theses and Dissertations UC Riverside Peer Reviewed Title: Bayesian and Non-parametric Approaches to Missing Data Analysis Author: Yu, Yao Acceptance Date: 01 Series: UC Riverside Electronic

More information

Bayesian Methods. 1 The Joint Posterior Distribution

Bayesian Methods. 1 The Joint Posterior Distribution Bayesian Methods Every variable in a linear model is a random variable derived from a distribution function. A fixed factor becomes a random variable with possibly a uniform distribution going from a lower

More information

Validation of Software for Bayesian Models using Posterior Quantiles. Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT

Validation of Software for Bayesian Models using Posterior Quantiles. Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT Validation of Software for Bayesian Models using Posterior Quantiles Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT Abstract We present a simulation-based method designed to establish that software

More information

CHAPTER 9 EXAMPLES: MULTILEVEL MODELING WITH COMPLEX SURVEY DATA

CHAPTER 9 EXAMPLES: MULTILEVEL MODELING WITH COMPLEX SURVEY DATA Examples: Multilevel Modeling With Complex Survey Data CHAPTER 9 EXAMPLES: MULTILEVEL MODELING WITH COMPLEX SURVEY DATA Complex survey data refers to data obtained by stratification, cluster sampling and/or

More information

11. Time series and dynamic linear models

11. Time series and dynamic linear models 11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

Introduction to Multilevel Modeling Using HLM 6. By ATS Statistical Consulting Group

Introduction to Multilevel Modeling Using HLM 6. By ATS Statistical Consulting Group Introduction to Multilevel Modeling Using HLM 6 By ATS Statistical Consulting Group Multilevel data structure Students nested within schools Children nested within families Respondents nested within interviewers

More information

Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu)

Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu) Paper Author (s) Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu) Lei Zhang, University of Maryland, College Park (lei@umd.edu) Paper Title & Number Dynamic Travel

More information

HLM software has been one of the leading statistical packages for hierarchical

HLM software has been one of the leading statistical packages for hierarchical Introductory Guide to HLM With HLM 7 Software 3 G. David Garson HLM software has been one of the leading statistical packages for hierarchical linear modeling due to the pioneering work of Stephen Raudenbush

More information

Longitudinal Meta-analysis

Longitudinal Meta-analysis Quality & Quantity 38: 381 389, 2004. 2004 Kluwer Academic Publishers. Printed in the Netherlands. 381 Longitudinal Meta-analysis CORA J. M. MAAS, JOOP J. HOX and GERTY J. L. M. LENSVELT-MULDERS Department

More information

IBM SPSS Missing Values 22

IBM SPSS Missing Values 22 IBM SPSS Missing Values 22 Note Before using this information and the product it supports, read the information in Notices on page 23. Product Information This edition applies to version 22, release 0,

More information

Gaussian Processes to Speed up Hamiltonian Monte Carlo

Gaussian Processes to Speed up Hamiltonian Monte Carlo Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo

More information

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could

More information

GLM, insurance pricing & big data: paying attention to convergence issues.

GLM, insurance pricing & big data: paying attention to convergence issues. GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK - michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.

More information

Sample Size Designs to Assess Controls

Sample Size Designs to Assess Controls Sample Size Designs to Assess Controls B. Ricky Rambharat, PhD, PStat Lead Statistician Office of the Comptroller of the Currency U.S. Department of the Treasury Washington, DC FCSM Research Conference

More information

A Bayesian hierarchical surrogate outcome model for multiple sclerosis

A Bayesian hierarchical surrogate outcome model for multiple sclerosis A Bayesian hierarchical surrogate outcome model for multiple sclerosis 3 rd Annual ASA New Jersey Chapter / Bayer Statistics Workshop David Ohlssen (Novartis), Luca Pozzi and Heinz Schmidli (Novartis)

More information

CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression

CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression Opening Example CHAPTER 13 SIMPLE LINEAR REGREION SIMPLE LINEAR REGREION! Simple Regression! Linear Regression Simple Regression Definition A regression model is a mathematical equation that descries the

More information

Applications of R Software in Bayesian Data Analysis

Applications of R Software in Bayesian Data Analysis Article International Journal of Information Science and System, 2012, 1(1): 7-23 International Journal of Information Science and System Journal homepage: www.modernscientificpress.com/journals/ijinfosci.aspx

More information

International Statistical Institute, 56th Session, 2007: Phil Everson

International Statistical Institute, 56th Session, 2007: Phil Everson Teaching Regression using American Football Scores Everson, Phil Swarthmore College Department of Mathematics and Statistics 5 College Avenue Swarthmore, PA198, USA E-mail: peverso1@swarthmore.edu 1. Introduction

More information

Bayesian Statistical Analysis in Medical Research

Bayesian Statistical Analysis in Medical Research Bayesian Statistical Analysis in Medical Research David Draper Department of Applied Mathematics and Statistics University of California, Santa Cruz draper@ams.ucsc.edu www.ams.ucsc.edu/ draper ROLE Steering

More information

Least Squares Estimation

Least Squares Estimation Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David

More information

UNIT 1: COLLECTING DATA

UNIT 1: COLLECTING DATA Core Probability and Statistics Probability and Statistics provides a curriculum focused on understanding key data analysis and probabilistic concepts, calculations, and relevance to real-world applications.

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

More information

College Readiness LINKING STUDY

College Readiness LINKING STUDY College Readiness LINKING STUDY A Study of the Alignment of the RIT Scales of NWEA s MAP Assessments with the College Readiness Benchmarks of EXPLORE, PLAN, and ACT December 2011 (updated January 17, 2012)

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Contents. List of Figures. List of Tables. List of Examples. Preface to Volume IV

Contents. List of Figures. List of Tables. List of Examples. Preface to Volume IV Contents List of Figures List of Tables List of Examples Foreword Preface to Volume IV xiii xvi xxi xxv xxix IV.1 Value at Risk and Other Risk Metrics 1 IV.1.1 Introduction 1 IV.1.2 An Overview of Market

More information

Analyzing Intervention Effects: Multilevel & Other Approaches. Simplest Intervention Design. Better Design: Have Pretest

Analyzing Intervention Effects: Multilevel & Other Approaches. Simplest Intervention Design. Better Design: Have Pretest Analyzing Intervention Effects: Multilevel & Other Approaches Joop Hox Methodology & Statistics, Utrecht Simplest Intervention Design R X Y E Random assignment Experimental + Control group Analysis: t

More information

A Guide to Sample Size Calculations for Random Effect Models via Simulation and the MLPowSim Software Package

A Guide to Sample Size Calculations for Random Effect Models via Simulation and the MLPowSim Software Package A Guide to Sample Size Calculations for Random Effect Models via Simulation and the MLPowSim Software Package William J Browne, Mousa Golalizadeh Lahi* & Richard MA Parker School of Clinical Veterinary

More information

Simple Linear Regression Inference

Simple Linear Regression Inference Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

More information

The Exponential Family

The Exponential Family The Exponential Family David M. Blei Columbia University November 3, 2015 Definition A probability density in the exponential family has this form where p.x j / D h.x/ expf > t.x/ a./g; (1) is the natural

More information

Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014 Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

More information

Electronic Thesis and Dissertations UCLA

Electronic Thesis and Dissertations UCLA Electronic Thesis and Dissertations UCLA Peer Reviewed Title: A Multilevel Longitudinal Analysis of Teaching Effectiveness Across Five Years Author: Wang, Kairong Acceptance Date: 2013 Series: UCLA Electronic

More information

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

More information

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

More information

Dealing with large datasets

Dealing with large datasets Dealing with large datasets (by throwing away most of the data) Alan Heavens Institute for Astronomy, University of Edinburgh with Ben Panter, Rob Tweedie, Mark Bastin, Will Hossack, Keith McKellar, Trevor

More information

Notes on Factoring. MA 206 Kurt Bryan

Notes on Factoring. MA 206 Kurt Bryan The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor

More information

The Proportional Odds Model for Assessing Rater Agreement with Multiple Modalities

The Proportional Odds Model for Assessing Rater Agreement with Multiple Modalities The Proportional Odds Model for Assessing Rater Agreement with Multiple Modalities Elizabeth Garrett-Mayer, PhD Assistant Professor Sidney Kimmel Comprehensive Cancer Center Johns Hopkins University 1

More information

Copulas in Financial Risk Management Dr Jorn Rank Department for Continuing Education University of Oxford A thesis submitted for the diploma in Mathematical Finance 4 August 2000 Diploma in Mathematical

More information

Models for Longitudinal and Clustered Data

Models for Longitudinal and Clustered Data Models for Longitudinal and Clustered Data Germán Rodríguez December 9, 2008, revised December 6, 2012 1 Introduction The most important assumption we have made in this course is that the observations

More information

Bayesian Statistics in One Hour. Patrick Lam

Bayesian Statistics in One Hour. Patrick Lam Bayesian Statistics in One Hour Patrick Lam Outline Introduction Bayesian Models Applications Missing Data Hierarchical Models Outline Introduction Bayesian Models Applications Missing Data Hierarchical

More information