Using SAS PROC MCMC to Estimate and Evaluate Item Response Theory Models

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Using SAS PROC MCMC to Estimate and Evaluate Item Response Theory Models"

Transcription

1 Using SAS PROC MCMC to Estimate and Evaluate Item Response Theory Models Clement A Stone Abstract Interest in estimating item response theory (IRT) models using Bayesian methods has grown tremendously in recent years. In part this is due to the appeal of the Bayesian paradigm among psychometricians and statisticians, as well as the advantages of these methods with small sample sizes and for more complex or highly parameterized models (e.g., multidimensional IRT models). Recently, routines have become available in SAS to also implement general Bayesian estimation methods (PROC MCMC). The purpose of this paper is to illustrate the use of PROC MCMC to estimate and evaluate the fit of IRT models, as well as to compare usage with WinBUGS. Examples will be presented that illustrate the syntax used to estimate the models (simple and more complex IRT models), and output will be presented to illustrate the type of output that is available to evaluate the convergence of the MCMC algorithm. Finally, tools for comparing competing models will be discussed as well as use of the posterior predictive distribution to evaluate model fit. 1 Item Response Theory Models Psychometricians (c.f., Hambleton and Swaminathan, 1985; Embretson and Reise, 2000) have discussed the advantages of applying item response theory (IRT) as opposed to classical test theory (CTT) to scale development. IRT consists of a family of models which can be used to predict performance on each item, or the probability of choosing a response category, using characteristics of items and persons. More specifically, IRT models consist of one or more latent ability/trait parameters (person characteristics) and different item parameters (Embretson and Reise, 2000; Hambleton and Swaminathan, 1985). These parameters in conjunction with a mathematical function (normal ogive or logistic functions) may be used to model the probability of a response (e.g., correct/incorrect, or endorsing a particular response option): Clement A Stone, University of Pittsburgh;

2 2 Clement A Stone P j (U j = u j ω j, θ) where, θ is a potentially multidimensional vector of person characteristics (e.g. abilities or traits being measured) and ω j is a vector of item characteristics included in the item response function for the IRT model. Different IRT models can be applied to assessments that vary in terms of the scoring options as well as the number of traits being measured. For, dichotomously scored items standard IRT models may be used such as the Rasch or 1-Parmaeter, 2- Parameter, and 3-Parameter IRT models (Hambleton and Swaminathan, 1985). For polytomously scored items such as with ordinal rating scales, the graded response (GR) IRT model (Samejima, 1969) or polytomous extensions of Rasch or 1-Parameter IRT models (Muraki,1992) may be used. Multi-dimensional extensions of these IRT models (compensatory and bi-factor models) incorporate additional person parameters to reflect multiple traits being measured, and additional item parameters to reflect the extent to which each item is related to each dimension (c.f., Muraki and Carlson, 1995). Also, when local dependence (LD) among a set of items exists, the dependency within items can be modelled within a modified IRT model (e.g., Wang, Bradlow, and Wainer 2002). These latter models are more highly parameterized which complicates the estimation process, and may demand alternative methods for estimating parameters. The purpose of this paper is to illustrate Bayesian estimation of a subset of these models using SAS PROC MCMC. The models illustrated include unidimensional and multidimension IRT models for dichotomously scored items. It should be noted that use of SAS PROC MCMC is not limited to these models. As will be discussed all that is necessary to estimate other models is to change either the models or likelihood equations that are defined to reflect desired parameterizations. 2 Bayesian Estimation of IRT Models In contrast to traditional statistics, a Bayesian paradigm considers model parameters to be random variables and is used to obtain distributions for model parameters from which direct probability statements about parameters may be made. Bayesian analysis begins with an assessment about the uncertainty in a parameter (prior distribution) before data is collected. Using Bayes theorem, a distribution for the parameter given the data (posterior distribution) may be obtained by updating beliefs about how parameters are distributed (prior distribution) with the distribution of data given the parameters (likelihood function:: p(θ D)= p(d θ) p(θ) / p(d) where p(d θ) is the likelihood function (Probability of data, D, given possible values of θ), p(θ) is the prior distribution for the parameter θ or prior belief or uncertainty about θ, and p(d) is the marginal or unconditional probability of data across all possible values of θ also described as the probability of data being generated by the statistical model. See Fox (2010) for a discussion of applications to IRT models. In addition to the useful interpretations afforded by a Bayesian paradigm, other advantages to using Bayesian methods to estimate IRT models include the following: Parameter estimates are less variable than Maximum Likelihood (ML) estimates, and are generally closer to true values in small samples and short tests

3 Using SAS PROC MCMC to Estimate and Evaluate IRT Models 3 Methods accommodate perfect and imperfect response patterns Ability or trait parameter estimates available simultaneously with item parameter estimates; therefore, uncertainty in item parameter estimates incorporated directly into ability or trait parameter estimates Provides a more stable estimation process; model parameter estimates do not drift out of bounds Can be used readily with complex or highly parameterized models (parameters that reflect additional facets of an assessment) For most models, Bayesian analysis and computing posterior distributions involves complex integrals for marginal probability distributions. As the number of model parameters increases, the set of integrals defined by p(d) is numerically intractable. To solve this problem, MCMC (Markov Chain Monte Carlo) methods may be used to generate a sample of parameter values from the joint posterior distributions for parameters. Thus, simulations are used to replace the complex mathematical calculations, and the availability of increased computing power has made Bayesian analysis via MCMC much more accessible. See, for example, Fox (2010) for a discussion of MCMC algorithms 3 Using SAS PROC MCMC to Estimate IRT Models Bayesian analysis by the Markov Chain Monte Carlo (MCMC) method has been greatly simplified through the WinBUGS computer program (Spiegelhalter, Thomas, and Best 2000). Recently, routines have become available in SAS to also implement general Bayesian estimation methods (PROC MCMC). SAS has several significant advantages over WinBUGS: 1) It is commonly used by researchers across disciplines; 2) It provides a robust programming language that extends the capability of the program; and 3) It displays increased performance and efficiency. As in WinBUGS, specification of the model is straightforward. All that is necessary is to specify the model or likelihood, model parameters, and prior distributions for model parameters. Information about prior distributions for parameters may be available from previous assessments, from the purpose of the assessment, or from the known properties of IRT response functions. For example, specification of prior distributions can consider whether an assessment serves as a broad measure of achievement rather than designed to measure the trait or ability in a local region of the trait or ability continuum. However, often vague or uninformative priors are defined to mitigate the influence of priors on the estimation process. For these types of priors, the probability distributions are relatively flat across the range of possible values for the parameter. 3.1 SAS PROC MCMC Template 1P IRT Model As a template for using PROC MCMC, the PROC MCMC command to estimate a 1- Parameter IRT model for 5 dichotomously scored items is as follows: PROC MCMC data=<item response dataset> outpost=<sas dataset for posterior distribution results>;

4 4 Clement A Stone array b[5]; array x[5]; array p[5]; * specify arrays for variables; parms a 1 b: 0; * specify initial values for item parameters; prior b: ~ normal(0, var=value) ; * define prior distributions for item parameters; prior a ~ lognormal(0, var=value); * specify prior distribution for trait parameters random effects in the model; random theta ~ normal(0, var=1) subject=id; * specify likelihood for IRT model; do j=1 to 5; p[j] = 1/(1+exp(-a*(theta - b[j]))); end; model x1 ~ binary(p1); model x2 ~ binary(p2); model x3 ~ binary(p3); model x4 ~ binary(p4); model x5 ~ binary(p5); Note that Normal and Lognormal priors are typically specified for the b and a parameters, respectively. Also, note that missing values are accommodated in PROC MCMC; any missing responses are treated as random variables and sampled directly using the response model. Alternatively, the above likelihood expression can be specified more generally as: llike = 0; do j=1 to 5; prob = 1/(1+exp(-a*(theta - b[j]))); llike = llike + x[j] * log(prob) + (1 - x[j]) * log(1 - prob); end; model general(llike); Such a specification allows for any likelihood for an IRT model to be specified. In particular, since no MODEL statement is available to accommodate non-binary responses, this specification is necessary for polytomously scored items. Since the MCMC method uses simulation procedures, it is important to determine if the Markov Chain has converged to the target posterior distribution. If the chain does not converge, the simulated draws from this chain would not represent the posterior distribution of interest. Therefore, it is very important to assess convergence of Markov chains before any making Bayesian inferences. As is the case with WinBUGS, numerous diagnostic tools are available to determine the extent to which the chain has converged. One popular tool are trace plots of sampled parameter values versus the iteration number in the chain. A chain that mixes well traverses the posterior space rapidly and reaches all regions of the relevant parameter space. In addition to trace plots, other commonly used statistics are available. Also available are a number of statistics for analysing the posterior distribution. These include the mean, standard deviation, percentiles, Bayesian equal-tail credible interval, highest posterior density (HPD) interval, and Deviance Information Criterion (DIC). DIC (Spiegelhalter et al., 2002) is a model comparison tool similar to the well known Akaike information criterion (AIC) and the Bayesian information criterion

5 Using SAS PROC MCMC to Estimate and Evaluate IRT Models 5 (BIC). DIC can be applied to nonnested models and models that have non-iid data. Smaller DIC values indicate better fit to the data. 4 Evaluating the Fit of IRT Models While other software is available that simplifies Bayesian estimation of IRT models (e.g., WinBugs), it is equally important to evaluate the fit of a particular IRT model to item responses. When a model doesn t fit the data, the validity of any inferences for the model parameter estimates will be threatened. SAS PROC MCMC affords advantages over other available software for Bayesian estimation of IRT models. For evaluating model-data-fit, in particular, SAS provides a robust programming language and built-in statistical/graphical tools, all of which expands the capability of the program beyond estimating the model to computing many different types of statistics for comparing competing models and evaluating model fit. The Posterior Predictive Model Checking (PPMC) method is a commonly used Bayesian model checking tool and has proved useful with IRT models (e.g., Levy, Mislevy, and Sinharay, 2009; Sinharay, 2006; Sinharay, Johnson, and Stern, 2006; Zhu and Stone, 2011). PPMC involves simulating data under the assumed or estimated model and comparing features of simulated data against observed data using discrepancy measures or statistics calculated on both the simulated and observed data (e.g., residual or total score). Specifically, PPMC assesses the fit of a model by examining whether the observed data, y, appears extreme with respect to data replicated from the model (y rep ): p rep rep rep ( y y ) = p( y, θ y) dθ p( y θ) p( θ y) dθ = The rationale underlying PPMC is that if a model fits the data, then observed data should be similar to replicated or simulated data based on the posterior distributions for model parameters (p(θ y)). In turn, distributions of discrepancy measures calculated on both simulated and observed data should also be similar if a model fits the data. As would be expected, choosing appropriate discrepancy measures is important when assessing model-fit using PPMC. Measures should be chosen to reflect potential sources of misfit most relevant to a particular application (Zhu and Stone, 2011; Zhu and Stone, in press). For example, when a unidimensional IRT model is estimated for item responses that may reflect a multidimensional structure, discrepancy measures should be chosen that are sensitive to this threat to model fit. 5 Presentation Format Examples will be presented that illustrate the syntax used to estimate the models (simple and more complex IRT models), and output will be presented to illustrate the type of diagnostics that are available to evaluate the convergence of the MCMC algorithm. Specifications of priors and how problems with convergence can be addressed will also be discussed. Finally, tools for comparing competing models will be

6 6 Clement A Stone discussed, as well as use of the posterior predictive distribution to evaluate model fit and the ease with which PPMC may be implemented in SAS PROC MCMC. Some of the discrepancy measures to be discussed will be based on Zhu s and Stone s (2011) research on Bayesian estimation of IRT models for performance assessments. References Embretson, S.E., Reise, S.P. Item response theory for psychologists. Mahwah: New Jersey: Lawrence Erlbaum. Fox, J.P. (2010). Bayesian item response modelling. NY: Springer. Hambleton, R. K., Swaminathan, H. (1985). Item response theory. Boston, MA: Kluwer-Nijhoff. Levy, R., Mislevy, R. J., Sinharay, S. (2009). Posterior predictive model checking for multidimensionality in item response theory. Appl. Psychol. Meas., 33(7), Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Appl. Psychol. Meas., 16, Muraki, E., Carlson, J. E. (1995). Full-information factor analysis for polytomous item responses. Appl. Psychol. Meas., 19, Samejima, F. (1969). Estimation of latent ability groups using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17. Sinharay, S. (2006). Bayesian item fit analysis for unidimensional item response theory models. Brit. J. Math. Stat. Psy., 59, Sinharay, S., Johnson, M. S., Stern, H. S. (2006). Posterior predictive assessment of item response theory models. Appl. Psychol. Meas., 30(4), Spiegelhalter, D.J., Thomas, A., Best, N. (2000). WinBUGS v1.3. Cambridge, UK: MRC Biostatistics Unit. Wang, X., Bradlow, E. T., Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Appl. Psychol. Meas., 26(1), Zhu, X., Stone, C.A. (In Press). Bayesian comparison of alternative graded response models for performance assessment applications. Educ. Psychol. Meas. Zhu, X., Stone, C.A. (2011). Assessing fit of unidimensional graded response models using Bayesian methods. J. Educ. Meas., 48,

Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It

Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It Paper SAS364-2014 Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It Xinming An and Yiu-Fai Yung, SAS Institute Inc. ABSTRACT Item response theory (IRT) is concerned with

More information

Lecture #2 Overview. Basic IRT Concepts, Models, and Assumptions. Lecture #2 ICPSR Item Response Theory Workshop

Lecture #2 Overview. Basic IRT Concepts, Models, and Assumptions. Lecture #2 ICPSR Item Response Theory Workshop Basic IRT Concepts, Models, and Assumptions Lecture #2 ICPSR Item Response Theory Workshop Lecture #2: 1of 64 Lecture #2 Overview Background of IRT and how it differs from CFA Creating a scale An introduction

More information

Lab 8: Introduction to WinBUGS

Lab 8: Introduction to WinBUGS 40.656 Lab 8 008 Lab 8: Introduction to WinBUGS Goals:. Introduce the concepts of Bayesian data analysis.. Learn the basic syntax of WinBUGS. 3. Learn the basics of using WinBUGS in a simple example. Next

More information

Keith A. Boughton, Don A. Klinger, and Mark J. Gierl. Center for Research in Applied Measurement and Evaluation University of Alberta

Keith A. Boughton, Don A. Klinger, and Mark J. Gierl. Center for Research in Applied Measurement and Evaluation University of Alberta Effects of Random Rater Error on Parameter Recovery of the Generalized Partial Credit Model and Graded Response Model Keith A. Boughton, Don A. Klinger, and Mark J. Gierl Center for Research in Applied

More information

Diagnostic Scores Augmented Using Multidimensional Item Response Theory: Preliminary Investigation of MCMC Strategies

Diagnostic Scores Augmented Using Multidimensional Item Response Theory: Preliminary Investigation of MCMC Strategies Diagnostic Scores Augmented Using Multidimensional Item Response Theory: Preliminary Investigation of MCMC Strategies David Thissen & Michael C. Edwards The University of North Carolina at Chapel Hill

More information

CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS

CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS Examples: Regression And Path Analysis CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS Regression analysis with univariate or multivariate dependent variables is a standard procedure for modeling relationships

More information

Imputing Missing Data using SAS

Imputing Missing Data using SAS ABSTRACT Paper 3295-2015 Imputing Missing Data using SAS Christopher Yim, California Polytechnic State University, San Luis Obispo Missing data is an unfortunate reality of statistics. However, there are

More information

Application of a Psychometric Rating Model to

Application of a Psychometric Rating Model to Application of a Psychometric Rating Model to Ordered Categories Which Are Scored with Successive Integers David Andrich The University of Western Australia A latent trait measurement model in which ordered

More information

Hierarchical Bayes Small Area Estimates of Adult Literacy Using Unmatched Sampling and Linking Models

Hierarchical Bayes Small Area Estimates of Adult Literacy Using Unmatched Sampling and Linking Models Hierarchical Bayes Small Area Estimates of Adult Literacy Using Unmatched Sampling and Linking Models Leyla Mohadjer 1, J.N.K. Rao, Benmei Liu 1, Tom Krenzke 1, and Wendy Van de Kerckhove 1 Leyla Mohadjer,

More information

Package EstCRM. July 13, 2015

Package EstCRM. July 13, 2015 Version 1.4 Date 2015-7-11 Package EstCRM July 13, 2015 Title Calibrating Parameters for the Samejima's Continuous IRT Model Author Cengiz Zopluoglu Maintainer Cengiz Zopluoglu

More information

Bayesian Statistical Analysis in Medical Research

Bayesian Statistical Analysis in Medical Research Bayesian Statistical Analysis in Medical Research David Draper Department of Applied Mathematics and Statistics University of California, Santa Cruz draper@ams.ucsc.edu www.ams.ucsc.edu/ draper ROLE Steering

More information

Handling attrition and non-response in longitudinal data

Handling attrition and non-response in longitudinal data Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

More information

Bayesian Statistics in One Hour. Patrick Lam

Bayesian Statistics in One Hour. Patrick Lam Bayesian Statistics in One Hour Patrick Lam Outline Introduction Bayesian Models Applications Missing Data Hierarchical Models Outline Introduction Bayesian Models Applications Missing Data Hierarchical

More information

An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics

An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Slide 1 An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Dr. Christian Asseburg Centre for Health Economics Part 1 Slide 2 Talk overview Foundations of Bayesian statistics

More information

Model Calibration with Open Source Software: R and Friends. Dr. Heiko Frings Mathematical Risk Consulting

Model Calibration with Open Source Software: R and Friends. Dr. Heiko Frings Mathematical Risk Consulting Model with Open Source Software: and Friends Dr. Heiko Frings Mathematical isk Consulting Bern, 01.09.2011 Agenda in a Friends Model with & Friends o o o Overview First instance: An Extreme Value Example

More information

Combining information from different survey samples - a case study with data collected by world wide web and telephone

Combining information from different survey samples - a case study with data collected by world wide web and telephone Combining information from different survey samples - a case study with data collected by world wide web and telephone Magne Aldrin Norwegian Computing Center P.O. Box 114 Blindern N-0314 Oslo Norway E-mail:

More information

11. Time series and dynamic linear models

11. Time series and dynamic linear models 11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd

More information

SAS Certificate Applied Statistics and SAS Programming

SAS Certificate Applied Statistics and SAS Programming SAS Certificate Applied Statistics and SAS Programming SAS Certificate Applied Statistics and Advanced SAS Programming Brigham Young University Department of Statistics offers an Applied Statistics and

More information

A Bayesian Approach to Person Fit Analysis in Item Response Theory Models

A Bayesian Approach to Person Fit Analysis in Item Response Theory Models A Bayesian Approach to Person Fit Analysis in Item Response Theory Models Cees A. W. Glas and Rob R. Meijer University of Twente, The Netherlands A Bayesian approach to the evaluation of person fit in

More information

Validation of Software for Bayesian Models using Posterior Quantiles. Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT

Validation of Software for Bayesian Models using Posterior Quantiles. Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT Validation of Software for Bayesian Models using Posterior Quantiles Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT Abstract We present a simulation-based method designed to establish that software

More information

Bayesian Methods. 1 The Joint Posterior Distribution

Bayesian Methods. 1 The Joint Posterior Distribution Bayesian Methods Every variable in a linear model is a random variable derived from a distribution function. A fixed factor becomes a random variable with possibly a uniform distribution going from a lower

More information

A General Diagnostic Model Applied to Language Testing Data

A General Diagnostic Model Applied to Language Testing Data Research Report A General Diagnostic Model Applied to Language Testing Data Matthias von Davier Research & Development September 2005 RR-05-16 A General Diagnostic Model Applied to Language Testing Data

More information

Goodness of fit assessment of item response theory models

Goodness of fit assessment of item response theory models Goodness of fit assessment of item response theory models Alberto Maydeu Olivares University of Barcelona Madrid November 1, 014 Outline Introduction Overall goodness of fit testing Two examples Assessing

More information

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics.

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics. Course Catalog In order to be assured that all prerequisites are met, students must acquire a permission number from the education coordinator prior to enrolling in any Biostatistics course. Courses are

More information

A New Item Selection Procedure for Mixed Item Type. in Computerized Classification Testing

A New Item Selection Procedure for Mixed Item Type. in Computerized Classification Testing A New Item Selection Procedure for Mixed Item Type in Computerized Classification Testing C. Allen Lau Harcourt Educational Measurement Tianyou Wang ACT Paper presented at the 2000 AERA Annual Meeting

More information

Chapter 1 Introduction. 1.1 Introduction

Chapter 1 Introduction. 1.1 Introduction Chapter 1 Introduction 1.1 Introduction 1 1.2 What Is a Monte Carlo Study? 2 1.2.1 Simulating the Rolling of Two Dice 2 1.3 Why Is Monte Carlo Simulation Often Necessary? 4 1.4 What Are Some Typical Situations

More information

Characteristics of Global Calling in VoIP services: A logistic regression analysis

Characteristics of Global Calling in VoIP services: A logistic regression analysis www.ijcsi.org 17 Characteristics of Global Calling in VoIP services: A logistic regression analysis Thanyalak Mueangkhot 1, Julian Ming-Sung Cheng 2 and Chaknarin Kongcharoen 3 1 Department of Business

More information

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could

More information

More details on the inputs, functionality, and output can be found below.

More details on the inputs, functionality, and output can be found below. Overview: The SMEEACT (Software for More Efficient, Ethical, and Affordable Clinical Trials) web interface (http://research.mdacc.tmc.edu/smeeactweb) implements a single analysis of a two-armed trial comparing

More information

THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE

THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE THE USE OF STATISTICAL DISTRIBUTIONS TO MODEL CLAIMS IN MOTOR INSURANCE Batsirai Winmore Mazviona 1 Tafadzwa Chiduza 2 ABSTRACT In general insurance, companies need to use data on claims gathered from

More information

Analyzing Clinical Trial Data via the Bayesian Multiple Logistic Random Effects Model

Analyzing Clinical Trial Data via the Bayesian Multiple Logistic Random Effects Model Analyzing Clinical Trial Data via the Bayesian Multiple Logistic Random Effects Model Bartolucci, A.A 1, Singh, K.P 2 and Bae, S.J 2 1 Dept. of Biostatistics, University of Alabama at Birmingham, Birmingham,

More information

Gaussian Processes to Speed up Hamiltonian Monte Carlo

Gaussian Processes to Speed up Hamiltonian Monte Carlo Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo

More information

Journal of Statistical Software

Journal of Statistical Software JSS Journal of Statistical Software October 2014, Volume 61, Issue 7. http://www.jstatsoft.org/ WebBUGS: Conducting Bayesian Statistical Analysis Online Zhiyong Zhang University of Notre Dame Abstract

More information

A Bayesian Model to Enhance Domestic Energy Consumption Forecast

A Bayesian Model to Enhance Domestic Energy Consumption Forecast Proceedings of the 2012 International Conference on Industrial Engineering and Operations Management Istanbul, Turkey, July 3 6, 2012 A Bayesian Model to Enhance Domestic Energy Consumption Forecast Mohammad

More information

Introduction to Bayesian Analysis Using SAS R Software

Introduction to Bayesian Analysis Using SAS R Software Introduction to Bayesian Analysis Using SAS R Software Joseph G. Ibrahim Department of Biostatistics University of North Carolina Introduction to Bayesian statistics Outline 1 Introduction to Bayesian

More information

Imperfect Debugging in Software Reliability

Imperfect Debugging in Software Reliability Imperfect Debugging in Software Reliability Tevfik Aktekin and Toros Caglar University of New Hampshire Peter T. Paul College of Business and Economics Department of Decision Sciences and United Health

More information

Incorporating cost in Bayesian Variable Selection, with application to cost-effective measurement of quality of health care.

Incorporating cost in Bayesian Variable Selection, with application to cost-effective measurement of quality of health care. Incorporating cost in Bayesian Variable Selection, with application to cost-effective measurement of quality of health care University of Florida 10th Annual Winter Workshop: Bayesian Model Selection and

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Item Response Theory in R using Package ltm

Item Response Theory in R using Package ltm Item Response Theory in R using Package ltm Dimitris Rizopoulos Department of Biostatistics, Erasmus University Medical Center, the Netherlands d.rizopoulos@erasmusmc.nl Department of Statistics and Mathematics

More information

LMM: Linear Mixed Models and FEV1 Decline

LMM: Linear Mixed Models and FEV1 Decline LMM: Linear Mixed Models and FEV1 Decline We can use linear mixed models to assess the evidence for differences in the rate of decline for subgroups defined by covariates. S+ / R has a function lme().

More information

Model-based Synthesis. Tony O Hagan

Model-based Synthesis. Tony O Hagan Model-based Synthesis Tony O Hagan Stochastic models Synthesising evidence through a statistical model 2 Evidence Synthesis (Session 3), Helsinki, 28/10/11 Graphical modelling The kinds of models that

More information

Applications of R Software in Bayesian Data Analysis

Applications of R Software in Bayesian Data Analysis Article International Journal of Information Science and System, 2012, 1(1): 7-23 International Journal of Information Science and System Journal homepage: www.modernscientificpress.com/journals/ijinfosci.aspx

More information

Validation of Software for Bayesian Models Using Posterior Quantiles

Validation of Software for Bayesian Models Using Posterior Quantiles Validation of Software for Bayesian Models Using Posterior Quantiles Samantha R. COOK, Andrew GELMAN, and Donald B. RUBIN This article presents a simulation-based method designed to establish the computational

More information

STATISTICA Formula Guide: Logistic Regression. Table of Contents

STATISTICA Formula Guide: Logistic Regression. Table of Contents : Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

More information

Bayesian Statistics: Indian Buffet Process

Bayesian Statistics: Indian Buffet Process Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note

More information

Markov Chain Monte Carlo Simulation Made Simple

Markov Chain Monte Carlo Simulation Made Simple Markov Chain Monte Carlo Simulation Made Simple Alastair Smith Department of Politics New York University April2,2003 1 Markov Chain Monte Carlo (MCMC) simualtion is a powerful technique to perform numerical

More information

A Latent Variable Approach to Validate Credit Rating Systems using R

A Latent Variable Approach to Validate Credit Rating Systems using R A Latent Variable Approach to Validate Credit Rating Systems using R Chicago, April 24, 2009 Bettina Grün a, Paul Hofmarcher a, Kurt Hornik a, Christoph Leitner a, Stefan Pichler a a WU Wien Grün/Hofmarcher/Hornik/Leitner/Pichler

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

Cognitive diagnostic assessment as an alternative measurement model by Vahid Aryadoust (National Institute of Education, Singapore) Abstract

Cognitive diagnostic assessment as an alternative measurement model by Vahid Aryadoust (National Institute of Education, Singapore) Abstract Cognitive diagnostic assessment as an alternative measurement model by Vahid Aryadoust (National Institute of Education, Singapore) Abstract Cognitive diagnostic assessment (CDA) is a development in psycho-educational

More information

A Bayesian Antidote Against Strategy Sprawl

A Bayesian Antidote Against Strategy Sprawl A Bayesian Antidote Against Strategy Sprawl Benjamin Scheibehenne (benjamin.scheibehenne@unibas.ch) University of Basel, Missionsstrasse 62a 4055 Basel, Switzerland & Jörg Rieskamp (joerg.rieskamp@unibas.ch)

More information

EC 6310: Advanced Econometric Theory

EC 6310: Advanced Econometric Theory EC 6310: Advanced Econometric Theory July 2008 Slides for Lecture on Bayesian Computation in the Nonlinear Regression Model Gary Koop, University of Strathclyde 1 Summary Readings: Chapter 5 of textbook.

More information

Computerized Adaptive Testing in Industrial and Organizational Psychology. Guido Makransky

Computerized Adaptive Testing in Industrial and Organizational Psychology. Guido Makransky Computerized Adaptive Testing in Industrial and Organizational Psychology Guido Makransky Promotiecommissie Promotor Prof. dr. Cees A. W. Glas Assistent Promotor Prof. dr. Svend Kreiner Overige leden Prof.

More information

Problem of Missing Data

Problem of Missing Data VASA Mission of VA Statisticians Association (VASA) Promote & disseminate statistical methodological research relevant to VA studies; Facilitate communication & collaboration among VA-affiliated statisticians;

More information

Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach

Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Refik Soyer * Department of Management Science The George Washington University M. Murat Tarimcilar Department of Management Science

More information

Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014 Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

More information

BAYESIAN ECONOMETRICS

BAYESIAN ECONOMETRICS BAYESIAN ECONOMETRICS VICTOR CHERNOZHUKOV Bayesian econometrics employs Bayesian methods for inference about economic questions using economic data. In the following, we briefly review these methods and

More information

Sample Size Designs to Assess Controls

Sample Size Designs to Assess Controls Sample Size Designs to Assess Controls B. Ricky Rambharat, PhD, PStat Lead Statistician Office of the Comptroller of the Currency U.S. Department of the Treasury Washington, DC FCSM Research Conference

More information

Lecture 6: The Bayesian Approach

Lecture 6: The Bayesian Approach Lecture 6: The Bayesian Approach What Did We Do Up to Now? We are given a model Log-linear model, Markov network, Bayesian network, etc. This model induces a distribution P(X) Learning: estimate a set

More information

Latent Trait Measurement Models for Binary Responses: IRT and IFA

Latent Trait Measurement Models for Binary Responses: IRT and IFA Latent Trait Measurement Models for Binary Responses: IRT and IFA Today s topics: The Big Picture of Measurement Models 1, 2, 3, and 4 Parameter IRT (and Rasch) Models Item and Test Information Item Response

More information

STAT3016 Introduction to Bayesian Data Analysis

STAT3016 Introduction to Bayesian Data Analysis STAT3016 Introduction to Bayesian Data Analysis Course Description The Bayesian approach to statistics assigns probability distributions to both the data and unknown parameters in the problem. This way,

More information

BAYESIAN ANALYSIS OF AN AGGREGATE CLAIM MODEL USING VARIOUS LOSS DISTRIBUTIONS. by Claire Dudley

BAYESIAN ANALYSIS OF AN AGGREGATE CLAIM MODEL USING VARIOUS LOSS DISTRIBUTIONS. by Claire Dudley BAYESIAN ANALYSIS OF AN AGGREGATE CLAIM MODEL USING VARIOUS LOSS DISTRIBUTIONS by Claire Dudley A dissertation submitted for the award of the degree of Master of Science in Actuarial Management School

More information

Electronic Thesis and Dissertations UCLA

Electronic Thesis and Dissertations UCLA Electronic Thesis and Dissertations UCLA Peer Reviewed Title: A Multilevel Longitudinal Analysis of Teaching Effectiveness Across Five Years Author: Wang, Kairong Acceptance Date: 2013 Series: UCLA Electronic

More information

Chapter 6: The Information Function 129. CHAPTER 7 Test Calibration

Chapter 6: The Information Function 129. CHAPTER 7 Test Calibration Chapter 6: The Information Function 129 CHAPTER 7 Test Calibration 130 Chapter 7: Test Calibration CHAPTER 7 Test Calibration For didactic purposes, all of the preceding chapters have assumed that the

More information

A hidden Markov model for criminal behaviour classification

A hidden Markov model for criminal behaviour classification RSS2004 p.1/19 A hidden Markov model for criminal behaviour classification Francesco Bartolucci, Institute of economic sciences, Urbino University, Italy. Fulvia Pennoni, Department of Statistics, University

More information

Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com

Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com Bayesian Machine Learning (ML): Modeling And Inference in Big Data Zhuhua Cai Google Rice University caizhua@gmail.com 1 Syllabus Bayesian ML Concepts (Today) Bayesian ML on MapReduce (Next morning) Bayesian

More information

Utilization of Response Time in Data Forensics of K-12 Computer-Based Assessment XIN LUCY LIU. Data Recognition Corporation (DRC) VINCENT PRIMOLI

Utilization of Response Time in Data Forensics of K-12 Computer-Based Assessment XIN LUCY LIU. Data Recognition Corporation (DRC) VINCENT PRIMOLI Response Time in K 12 Data Forensics 1 Utilization of Response Time in Data Forensics of K-12 Computer-Based Assessment XIN LUCY LIU Data Recognition Corporation (DRC) VINCENT PRIMOLI Data Recognition

More information

A Cautionary Note on Using G 2 (dif) to Assess Relative Model Fit in Categorical Data Analysis

A Cautionary Note on Using G 2 (dif) to Assess Relative Model Fit in Categorical Data Analysis MULTIVARIATE BEHAVIORAL RESEARCH, 4(), 55 64 Copyright 2006, Lawrence Erlbaum Associates, Inc. A Cautionary Note on Using G 2 (dif) to Assess Relative Model Fit in Categorical Data Analysis Albert Maydeu-Olivares

More information

Dependencies in Stochastic Loss Reserve Models

Dependencies in Stochastic Loss Reserve Models Glenn Meyers, FCAS, MAAA, Ph.D. Abstract Given a Bayesian Markov Chain Monte Carlo (MCMC) stochastic loss reserve model for two separate lines of insurance, this paper describes how to fit a bivariate

More information

Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data

Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data (Oxford) in collaboration with: Minjie Xu, Jun Zhu, Bo Zhang (Tsinghua) Balaji Lakshminarayanan (Gatsby) Bayesian

More information

A tutorial on Bayesian model selection. and on the BMSL Laplace approximation

A tutorial on Bayesian model selection. and on the BMSL Laplace approximation A tutorial on Bayesian model selection and on the BMSL Laplace approximation Jean-Luc (schwartz@icp.inpg.fr) Institut de la Communication Parlée, CNRS UMR 5009, INPG-Université Stendhal INPG, 46 Av. Félix

More information

Bayesian inference for population prediction of individuals without health insurance in Florida

Bayesian inference for population prediction of individuals without health insurance in Florida Bayesian inference for population prediction of individuals without health insurance in Florida Neung Soo Ha 1 1 NISS 1 / 24 Outline Motivation Description of the Behavioral Risk Factor Surveillance System,

More information

APPLIED MISSING DATA ANALYSIS

APPLIED MISSING DATA ANALYSIS APPLIED MISSING DATA ANALYSIS Craig K. Enders Series Editor's Note by Todd D. little THE GUILFORD PRESS New York London Contents 1 An Introduction to Missing Data 1 1.1 Introduction 1 1.2 Chapter Overview

More information

Imputing Values to Missing Data

Imputing Values to Missing Data Imputing Values to Missing Data In federated data, between 30%-70% of the data points will have at least one missing attribute - data wastage if we ignore all records with a missing value Remaining data

More information

Web-based Supplementary Materials for Bayesian Effect Estimation. Accounting for Adjustment Uncertainty by Chi Wang, Giovanni

Web-based Supplementary Materials for Bayesian Effect Estimation. Accounting for Adjustment Uncertainty by Chi Wang, Giovanni 1 Web-based Supplementary Materials for Bayesian Effect Estimation Accounting for Adjustment Uncertainty by Chi Wang, Giovanni Parmigiani, and Francesca Dominici In Web Appendix A, we provide detailed

More information

Government of Russian Federation. Faculty of Computer Science School of Data Analysis and Artificial Intelligence

Government of Russian Federation. Faculty of Computer Science School of Data Analysis and Artificial Intelligence Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education National Research University «Higher School of Economics» Faculty of Computer Science School

More information

Statistics Graduate Courses

Statistics Graduate Courses Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

More information

Appendix A: Sampling Methods

Appendix A: Sampling Methods Appendix A: Sampling Methods What is Sampling? Sampling is used in an @RISK simulation to generate possible values from probability distribution functions. These sets of possible values are then used to

More information

Lecture 3 : Hypothesis testing and model-fitting

Lecture 3 : Hypothesis testing and model-fitting Lecture 3 : Hypothesis testing and model-fitting These dark lectures energy puzzle Lecture 1 : basic descriptive statistics Lecture 2 : searching for correlations Lecture 3 : hypothesis testing and model-fitting

More information

The Effects of Testlets on Reliability and Differential Item Functioning *

The Effects of Testlets on Reliability and Differential Item Functioning * ISSN 1303-0485 eissn 2148-7561 DOI 10.12738/estp.2015.4.2577 Copyright 2015 EDAM http://www.estp.com.tr Educational Sciences: Theory & Practice 2015 August 15(4) 969-980 Received June 6, 2014 Accepted

More information

Time Series Analysis

Time Series Analysis Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Identification of univariate time series models, cont.:

More information

Tutorial on Markov Chain Monte Carlo

Tutorial on Markov Chain Monte Carlo Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,

More information

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis Int. Journal of Math. Analysis, Vol. 5, 2011, no. 1, 1-13 Review of the Methods for Handling Missing Data in Longitudinal Data Analysis Michikazu Nakai and Weiming Ke Department of Mathematics and Statistics

More information

PS 271B: Quantitative Methods II. Lecture Notes

PS 271B: Quantitative Methods II. Lecture Notes PS 271B: Quantitative Methods II Lecture Notes Langche Zeng zeng@ucsd.edu The Empirical Research Process; Fundamental Methodological Issues 2 Theory; Data; Models/model selection; Estimation; Inference.

More information

Ordinal Regression. Chapter

Ordinal Regression. Chapter Ordinal Regression Chapter 4 Many variables of interest are ordinal. That is, you can rank the values, but the real distance between categories is unknown. Diseases are graded on scales from least severe

More information

The Effect of Nonignorable Missing Data in Computerized Adaptive Test on Item Fit Statistics for Polytomous Item Response Models

The Effect of Nonignorable Missing Data in Computerized Adaptive Test on Item Fit Statistics for Polytomous Item Response Models The Effect of Nonignorable Missing Data in Computerized Adaptive Test on tem Fit Statistics for Polytomous tem Response Models Shudong Wang Northwest Evaluation Association Hong Jiao University of Maryland

More information

Centre for Central Banking Studies

Centre for Central Banking Studies Centre for Central Banking Studies Technical Handbook No. 4 Applied Bayesian econometrics for central bankers Andrew Blake and Haroon Mumtaz CCBS Technical Handbook No. 4 Applied Bayesian econometrics

More information

Item selection by latent class-based methods: an application to nursing homes evaluation

Item selection by latent class-based methods: an application to nursing homes evaluation Item selection by latent class-based methods: an application to nursing homes evaluation Francesco Bartolucci, Giorgio E. Montanari, Silvia Pandolfi 1 Department of Economics, Finance and Statistics University

More information

Common method bias in PLS-SEM: A full collinearity assessment approach

Common method bias in PLS-SEM: A full collinearity assessment approach Common method bias in PLS-SEM: A full collinearity assessment approach Ned Kock Full reference: Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal

More information

Measuring the tracking error of exchange traded funds: an unobserved components approach

Measuring the tracking error of exchange traded funds: an unobserved components approach Measuring the tracking error of exchange traded funds: an unobserved components approach Giuliano De Rossi Quantitative analyst +44 20 7568 3072 UBS Investment Research June 2012 Analyst Certification

More information

A Bayesian Computation for the Prediction of Football Match Results Using Artificial Neural Network

A Bayesian Computation for the Prediction of Football Match Results Using Artificial Neural Network A Bayesian Computation for the Prediction of Football Match Results Using Artificial Neural Network Mehmet Ali Cengiz 1 Naci Murat 2 Haydar Koç 3 Talat Şenel 4 1, 2, 3, 4 19 Mayıs University Turkey, Email:

More information

Growth, Precision, and CAT: An Examination of Gain Score Conditional SEM

Growth, Precision, and CAT: An Examination of Gain Score Conditional SEM Growth, Precision, and CAT: An Examination of Gain Score Conditional SEM Tony D. Thompson Research Report December 2008 Using assessment and research to promote learning Growth, Precision, and CAT 2 Abstract

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Gaussian Mixture Models Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique

More information

Basic Bayesian Methods

Basic Bayesian Methods 6 Basic Bayesian Methods Mark E. Glickman and David A. van Dyk Summary In this chapter, we introduce the basics of Bayesian data analysis. The key ingredients to a Bayesian analysis are the likelihood

More information

Analyzing Structural Equation Models With Missing Data

Analyzing Structural Equation Models With Missing Data Analyzing Structural Equation Models With Missing Data Craig Enders* Arizona State University cenders@asu.edu based on Enders, C. K. (006). Analyzing structural equation models with missing data. In G.

More information

Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities

Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities Oscar Kipersztok Mathematics and Computing Technology Phantom Works, The Boeing Company P.O.Box 3707, MC: 7L-44 Seattle, WA 98124

More information

Stephen du Toit Mathilda du Toit Gerhard Mels Yan Cheng. LISREL for Windows: PRELIS User s Guide

Stephen du Toit Mathilda du Toit Gerhard Mels Yan Cheng. LISREL for Windows: PRELIS User s Guide Stephen du Toit Mathilda du Toit Gerhard Mels Yan Cheng LISREL for Windows: PRELIS User s Guide Table of contents INTRODUCTION... 1 GRAPHICAL USER INTERFACE... 2 The Data menu... 2 The Define Variables

More information

How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power

How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power STRUCTURAL EQUATION MODELING, 9(4), 599 620 Copyright 2002, Lawrence Erlbaum Associates, Inc. TEACHER S CORNER How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power Linda K. Muthén

More information

Figure 1. IBM SPSS Statistics Base & Associated Optional Modules

Figure 1. IBM SPSS Statistics Base & Associated Optional Modules IBM SPSS Statistics: A Guide to Functionality IBM SPSS Statistics is a renowned statistical analysis software package that encompasses a broad range of easy-to-use, sophisticated analytical procedures.

More information

EMERGENCE OF IRT AND CAT IN HEALTH OUTCOMES MEASUREMENT PETER TEN KLOOSTER

EMERGENCE OF IRT AND CAT IN HEALTH OUTCOMES MEASUREMENT PETER TEN KLOOSTER EMERGENCE OF IRT AND CAT IN HEALTH OUTCOMES MEASUREMENT PETER TEN KLOOSTER HEALTH OUTCOMES MEASUREMENT Outcome measures provide an indication of the severity of the condition and the effects of treatment

More information

The Assumption(s) of Normality

The Assumption(s) of Normality The Assumption(s) of Normality Copyright 2000, 2011, J. Toby Mordkoff This is very complicated, so I ll provide two versions. At a minimum, you should know the short one. It would be great if you knew

More information