# Why High-Order Polynomials Should Not be Used in Regression Discontinuity Designs

Save this PDF as:

Size: px
Start display at page:

Download "Why High-Order Polynomials Should Not be Used in Regression Discontinuity Designs"

## Transcription

1 Why High-Order Polynomials Should Not be Used in Regression Discontinuity Designs Andrew Gelman Guido Imbens 2 Aug 2014 Abstract It is common in regression discontinuity analysis to control for high order (third, fourth, or higher) polynomials of the forcing variable. We argue that estimators for causal effects based on such methods can be misleading, and we recommend researchers do not use them, and instead use estimators based on local linear or quadratic polynomials or other smooth functions. Keywords: identification, policy analysis, polynomial regression, regression discontinuity, uncertainty 1. Introduction 1.1. Controlling for the forcing variable in regression discontinuity analysis Causal inference is central to science, and identification strategies in observational studies are central to causal inference in aspects of social and natural sciences when experimentation is not possible. Regression discontinuity designs (RDD) are a longstanding (going back to Thistlewaite and Campbell, 1960), and recently popular, way to get credible causal estimates when applicable. But implementations of regression discontinuity inference vary considerably in the literature, with many researchers controlling for high-degree polynomials of the underlying continuous forcing variable. In this note we make the case that high-order polynomial regressions have poor properties and argue that they should not be used in these settings. There are three, somewhat related, reasons why we think that high-order polynomial regressions are a poor choice in regression discontinuity analysis: 1. An estimate based on a polynomial regression, with or without trimming, can be interpreted as the difference between a weighted average of the outcomes for the treated and a weighted average for the controls. Given the choice of estimator, the weights depend only on the threshold and the values of the forcing variable, not on the values for the outcomes. One can, and should in applications, inspect these weights. We find that doing so in some applications suggests that the weights implied by higher-order polynomial regressions are not attractive compared to those for local linear regressions. We thank Jennifer Hill and Joseph Cummins for helpful comments and the National Science Foundation and Institute for Education Sciences for partial support of this work Department of Statistics, Columbia University, New York. Graduate School of Business, Stanford University, and NBER.

2 2. Results based on high order polynomial regressions are sensitive to the order of the polynomial. Moreover, we do not have good methods for choosing that order in a way that is optimal for the objective of a good estimator for the causal effect of interest. Often researchers choose the order by optimizing some global goodness of fit measure, but that is not closely related to the research objective of causal inference. 3. Inference based on high-order polynomials is often poor. Specifically, confidence intervals based on such regressions, taking them as accurate approximations to the regression function, are often misleading. Even if there is no discontinuity in the regression function, high-order polynomial regressions often lead to confidence intervals that fail to include zero with probability substantially higher than the nominal Type 1 error rate. Based on these arguments we recommend that researchers not use such methods, and instead control for local linear or quadratic polynomials or other smooth functions Theoretical framework Regression discontinuity analysis has analysis enjoyed a renaissance in social science, especially in economics; Imbens and Lemieux (2008), Van Der Klaauw (2013), Lee and Lemieux (2010) and DiNardo and Lee (2010) provide recent reviews. Regression discontinuity analyses is used to estimate the causal effect of a binary treatment on some outcome. Let (Y i (0), Y i (1)) denoted the pair of potential outcomes for unit i, and let W i {0, 1} denote the treatment. The realized outcome is Yi obs = Y i (W i ). Although the same issues arise in fuzzy regression discontinuity designs, for ease of exposition we focus on the sharp case where the treatment is a deterministic function of a pretreatment forcing variable X i : W i = 1 Xi 0. Define τ(x) = E[Y i (1) Y i (0) X i = x]. Regression discontinuity methods focus on estimating the average effect of the treatment at the threshold (equal to zero here): τ = τ(0). Under some conditions, mainly smoothness of the conditional expectations of the potential outcomes as a function of the forcing variable, this average effect can be estimated as the discontinuity in the conditional expectation of Yi obs as a function of the forcing variable, at the threshold: τ = lim E[Y i X i = x] lim E[Y i X i = x]. x 0 x 0 The question is how to estimate the two limits of the regression function at the threshold: µ + = lim x 0 E[Y i X i = x], and µ = lim x 0 E[Y i X i = x]. We focus in this note on two approaches researchers have commonly taken to estimating µ + and µ. Typically researchers are not confident that the two conditional means µ + (x) = 2

3 E[Y obs i X i = x, x > 0] and µ (x) = E[Yi obs X i = x, x < 0] can be well approximated by a global linear function. One approach researchers have taken is to use a global high-order polynomial approach. In this approach, researchers choose some integer K, and estimate the regression function, Y obs i = K Xi k β +j + ε +i, k=0 separately on the units with values for X i 0 and units with values for X i < 0. The discontinuity in the value of the regression function at zero is then estimated as, ˆτ = ˆµ + ˆµ = ˆβ +0 ˆβ 0. In practice, researchers often use cubic or higher order polynomials, often using statistical information criteria or cross-validation to determine choose the degree K of the polynomial. The second commonly-used approach is local linear or sometimes local quadratic approximation. In that case researchers discard the units with X i more than some bandwidth h away from the threshold and estimate a linear or quadratic function on the remaining units; see Hahn, Todd, and Van Der Klaauw (2001) and Porter (2003). Imbens and Kalyanaraman (2012) suggest a data driven way for choosing the bandwidth in connection with a local linear specification. Calonico, Cattaneo, and Titiunik (2014) suggest using a quadratic specification. The main point of the current paper is that we think the approach based on high order global polynomial approximations should not be used, and that instead inference based on local low order polynomials is to be preferred. In the next three sections we discuss three arguments in support of this position and illustrate these in the context of some applications. 2. Issue 1: Noisy weights Our first argument against using global high-order polynomial methods focuses on the interpretation of linear estimators for the causal estimand as weighting averages. More precisely, these estimators can be written as the difference between the weighted averages of the outcomes for the treated and controls, with the weights a function of the forcing variable. We show that for global polynomial methods these weights can have unattractive properties The weighted average representation of polynomial regressions The starting point is that polynomial regressions, whether global or local, lead to estimators for µ + and µ that can be written as weighted averages. Focusing on ˆµ +, the estimator for µ +, we can write ˆµ + as a weighted average of outcomes for units with X i 0: ˆµ + = 1 N + i:x i 0 ω i, Y obs i, where the weights ω i have been normalized to have a mean of 1 over all units with a value of the forcing variable exceeding zero. The weights are an estimator-specific function of the 3

4 full set of values X 1,..., X N for the forcing variable that does not depend on the outcome values Y1 obs,..., Y obs. Hence we can write the weights as N ω i = ω(x 1,..., X N ). The various estimators differ in the way the weights depend on value of the forcing variable. Moreover, we can inspect, for a given estimator, the functional form for the weights. Suppose we estimate a K-th order polynomial approximation using all units with X i less than the bandwidth h (where h can be so that this includes global polynomial regressions). Then the weight for unit i in the estimation of µ +, ˆµ + = i:x i 0 ω iyi obs /N +, is ω i = 1 0 Xi <h e K1 j:0 X i <h 1 X j... Xj K X j Xj 2... X K+1 j..... Xj K X K+1 j... Xj 2K where e K1 is the K-component column vector with all elements other than the first equal to 0, and the first element equal to 1. There are two important features of these weights. First, the values of the weights have nothing to do with the actual shape of the regression function, whether it is constant, linear, or anything else. Second, one can inspect these weights based on the values of the forcing variable in the sample, and compare them for different estimators. In particular we can compare, before seeing the outcome data, the weights for different values of the bandwidth h and the order of the polynomial K Example: Matsudaira data 1 To illustrate, we inspect the weights for various estimators for an analysis by Matsudaira (2008) of the effect of a remedial summer program on subsequent academic achievement. Students were required to participate in the summer program if they score below a threshold on either a mathematics or a reading test, although not all students did so, making this a fuzzy regression discontinuity design. We focus here on the discontinuity in the outcome variable, which can be interpreted as an intention-to-treat estimate. There are 68,798 students in the sample. The forcing variable is the minimum of the mathematics and reading test scores normalized so that the threshold equals 0. The outcome we look at here is the subsequent mathematics score. There are 22,892 students with the minimum of the test scores below the threshold, and 45,906 with a test score above. In this section we discuss estimation of µ + only. Estimation of µ raises the same issues. We look at weights for various estimators, without using the outcome data. First we consider global polynomials up to sixth degree. Next we consider local linear methods. The bandwidth for the local linear regression is 27.6, calculated using the Imbens and Kalyanaraman (2012) bandwidth selector. This leaves 22,892 individuals whose value for the forcing variable is positive and less than 27.6, out of the 45,906 with positive values for the forcing variable. We estimate the local linear regression using a triangular kernel. Figure 1 and Table 2.2 present some of the results relevant for the discussion on the weights. Figure 1a gives the weights for the six global polynomial regressions. Figure 1b 4 1 X i. X K i,

5 10 Figure 1a. weights for higher order polynomials Figure 1.b Weight Functions for Local Linear Estimator with Rectancular and Triangular Kernel, Matsudaira Data Figure 1c. Histogram of Forcing Variable Greater than Threshold, Matsudaira Data Figure 1: Weights, as a function of the forcing variable X, for persons in the Matsudaira dataset with positive values of X (a) weights for fitted polynomial regressions from first to sixth-order: the weights oscillate uncomfortably and are highly variable; (b) weights for local linear estimators are much better behaved; (c) histogram of the positive values of X. Higher-order regressions can give huge weights to points that are far from the discontinuity, thus creating highly noisy estimates of the causal estimand of interest. gives the weights for the local linear regression with rectangular and triangular kernels, and the bandwidth equal to Figure 1c presents a histogram of the distribution of the forcing variable for individuals with a value for the forcing variable greater than 0. In Table we present the weight, under each of the six polynomial regression specifications, for the individuals with the largest value for the forcing variable, X i = 168. Because this extreme value is outside the bandwidth, the weight for the local linear regression for individuals with such a value for X i would be 0. The weights for all the points included in the regression have been normalized so that their average value is 1. Figure 1a shows that the weight for the individuals with the largest values for the forcing variable are high in absolute value and highly sensitive to the order of the polynomial. It is difficult to imagine that, based on these figures, one would be comfortable with any of these six specifications. Figure 1b shows the weights for the local linear regression, which appear 5

6 Order of Normalized weight for the persons with global polynomial the largest value of the forcing variable Table 1: For different orders of the global polynomial: Normalized weight for the persons with the largest value of the forcing variable in the Matsudaira data. The weights are highly variable and all are much larger in absolute value than the average weight of 1. Considering that this point is far away from the discontinuity, this suggests that these polynomial regressions are assigning much too high a weight to points in the data that are essentially irrelevant to the research question. It would be more desirable for the weight of such an extreme point to be close to zero. more attractive: most of the weight goes to the individuals with values for X i close to the threshold, and individuals with X i > 27.6 have weights of 0. Table 1 also shows the unattractiveness of the high order polynomial regressions. Whereas one would like to give little or zero weight to the individuals with extreme values for X i, the global polynomial regressions attach large weights, sometimes positive, sometimes negative, to these individuals, and generally larger than the average weight of 1, whereas the local linear estimator attaches zero weight to these individuals General recommendation Most, if not all, estimators for average treatment effects used in practice can be written as the difference between two weighted averages, one for the treated units and one for the control units. This includes estimators in regression discontinuity settings. In those cases it is useful to inspect the weights in the weighted average expression for the estimators to assess whether some units receive excessive weight in the estimators. 3. Issue 2: Estimates that are highly sensitive to the degree of the polynomial The second argument against the high order global polynomial regressions is their sensitivity to the order of the polynomial. We illustrate that here using two applications of regression discontinuity designs. 6

7 Order of polynomial estimate (se) global (0.008) global (0.010) global (0.011) global (0.013) global (0.016) global (0.018) local (0.012) local (0.017) Table 2: From the Matsudaira data: Estimates of effect of summer school requirement using different regression discontinuity estimates Example: Matsudaira data (continued) We return to the Matsudaira data. Here we use the outcome data and directly estimate the effect of the treatment on the outcome for units close to the threshold. To simplify the exposition, we look at the effect of being required to attend summer school, rather than actual attendance, analyzing the data as a sharp rather than a fuzzy regression discontinuity design. We consider global polynomials up to order six and local polynomials up to order two. The bandwidth is 27.6 for the local polynomial estimators, based on the Imbens- Kalyanaraman bandwidth selector, leaving 37,580 in the sample. Local regression is with a triangular kernel. Table 2 displays the point estimates and standard errors. The variation in the global polynomial estimates over the six specifications is much bigger than the standard error for any of these six estimates. That the standard errors do not capture the full amount of uncertainty. The estimates based on third, fourth, fifth, and sixth order global polynomials range from to 0.112, whereas the range for the local linear and quadratic estimates is to 0.080, substantially narrower Example: Jacob-Lefgren data Table 3 reports the corresponding estimates for a second dataset. Here the interest is again in the causal effect of a summer school program. The data were previously analyzed by Jacob and Lefgren (2004). There are observations on 70,831 students. The forcing variable is the minimum of a mathematics and reading test. Out of the 70,831 students, 29,900 score below the threshold on at least one of the tests, and so are required to participate in the summer program. The Imbens-Kalyanaraman bandwidth here is As a result the local polynomial estimators are based on 31,747 individuals out of the full sample of 70,831, with 16,011 required and 15,736 not required to participate in summer school. Again the estimates based on the global polynomials have a wider range than the local linear and quadratic estimates. In addition the range for the global polynomial estimates is again large compared to the standard errors. 7

8 Order of polynomial estimate (se) global (0.008) global (0.012) global (0.015) global (0.018) global (0.021) global (0.025) local (0.018) local (0.027) Table 3: From the Jacob-Lefgren data: Estimates of the effect of summer school requirement using different regression discontinuity estimates. Linear regression 6th order polynomial Predictor estimate (se) estimate (se) intercept 5.0 (0.1) 3.6 (0.3) X 7.1 (0.1) 27.6 (2.7) X 2 54 (11) X 3 59 (18) X 4 32 (13) X 5 8 (5) X (0.6) Table 4: From the Lalonde data: Parameter estimates for linear and sixthorder polynomial regression of earnings in 1978 on the average of earnings in 1974 and Issue 3: Inferences that do not achieve nominal coverage The third point we wish to make is that conventional inference for treatment effects in regression discontinuity settings can be misleading, in the sense that that confidence intervals are too narrow. We make that point by constructing confidence intervals for discontinuities in a setting where we expect none to be present. We look at a dataset containing information on yearly earnings in 1974, 1975, and 1978 for 15,992 individuals for whom there is information from the Current Population Survey. These data were previously used for different purposes in work by Lalonde (1986) and Dehejia and Wahba (1999). We look at the conditional expectation of earnings in 1978 (the outcome Y i ) given the average of earnings in 1974 and 1975 (with the predictor X i divided by 10 for convenience so that the coefficients of the higher powers are on a reasonable scale). Figure 2 gives a simple, histogram-based estimate of the conditional expectation. Unsurprisingly, the conditional expectation looks fairly smooth and increasing. Overlaid with the histogram estimator are a linear approximation and a sixth order polynomial approximation. The linear approximation is fairly accurate, and so is the sixth order polynomial approximation. We 8

9 25 Fig 2: Lalonde Data, Regression of Earnings in 1978 on Average of Earnings in 1974, Earnings in Average of Earnings in 1974, 1975 Figure 2: From the Lalonde data: Earnings in 1978 vs. average earnings in We will use these data to fit a regression discontinuity estimate in a case where there is no actual discontinuity. present the parameter estimates for those two regression functions in Table 4. Now suppose we pretend the median of the average of earnings in 1974 and 1975 (equal to 14.65) was the threshold, and we estimate the discontinuity in the conditional expectation of earnings in We should expect to find an estimate close to zero. The estimates and standard errors for polynomials of different degree appear in Table 5. All estimates are in fact reasonably close to zero, with the nominal 95% confidence interval in all cases including zero, though this does not hold for the nominal 90% confidence interval. The question arises whether this is typical. To assess this we do the following exercise. 5,000 times we randomly pick a single point from the empirical distribution of X i that will serve as a pseudo threshold. We exclude values for the threshold less than 1 and greater than 25, to ensure a sufficient number of observations on both sides of the threshold. 1 We pretend this randomly drawn value of X i is the threshold in a regression discontinuity design analysis. In each of the 5,000 replications we then estimate the average effect of the pseudo treatment, its standard error, and check whether the implied 95% confidence interval excludes zero. There is no reason to expect a discontinuity in this conditional expectation at this threshold, and so we would like to see that only 5% of the times we randomly pick a threshold the corresponding confidence interval should not include zero. We compare the eight different estimates of the pseudo treatment effect we used in the previous section. The first six are global polynomial regressions of order ranging from 1 to 1 Out of the sample of 15,992 individuals there are 2044 individuals with values for X less than 1 and 2747 individuals with values for X greater than 25. 9

10 Order of polynomial estimate (se) global (0.25) global (0.25) global (0.53) global (0.66) global (0.80) global (0.93) local (0.37) local (0.55) Table 5: Estimates of effect of a pseudo treatment: Single replication on Lalonde Data with pseudo threshold equal to Order of Rejection Median polynomial rate standard error global global global global global global local local Table 6: Rejection rates for nominal 5% test in a simulation experiment in which the true discontinuity effect is zero. The over-rejection of the global polynomial regressions suggest that they are not accurate enough to allow the researcher to ignore the bias in the estimates of the treatment effects. 6. If, say, the regression functions on both sides of the threshold are truly linear, than the estimator based on linearity should be approximately unbiased for the average treatment effect (which is zero here), and the corresponding 95% confidence interval should include zero 95% of the time. If, on the other hand, the regression function is not truly linear, the confidence intervals based on linearity are likely to include the true value of zero less than 95% of the time. The last two estimators are based on local linear and local quadratic regressions, where we drop observations with values of X more than h away from the threshold (where the distance h is chosen using the Imbens-Kalyanaram bandwidth procedure). 2 We present the results for the rejection rates and median standard errors in Table 6. The estimates based on the global polynomial regressions perform poorly, making our point that inference for estimates based on global polynomial regressions is not reliable. For low order 2 In 50,000 replications the median bandwidth was

11 polynomials we reject the null of no effect for a large fraction of the replications. For high order global polynomials we still over-reject, and in addition the standard errors tend to be large compared to those for the local polynomial estimators. The over-rejection suggests that the global polynomial approximations are not accurate enough to allow the researcher to ignore the bias in the estimates of the treatment effects. The local linear and quadratic regressions work substantially better in that the rejection rates are close to nominal levels, and the standard errors are substantially smaller than those based on the high order global polynomial approximations. 5. Discussion Regression discontinuity designs have become very popular in social sciences in the last twenty years. One implementation relies on using high-order polynomial approximations to the conditional expectation of the outcome given the forcing variable. In this paper we recommend against using this method. We present three arguments for this position: the implicit weights for high order polynomial approximations are not attractive, the results are sensitive to the order of the polynomial approximation, and conventional inference has poor properties in these settings. These issues are complementary, in that the noisiness of the implicit weights explains how the global polynomial regressions can have poor coverage and wide confidence intervals at the same time. In addition we recommend that researchers routine present the implicit weights arising from regression estimates of causal estimands. 6. References Calonico, S., Cattaneo, M. D., and Titiunik, R. (2014). Robust nonparametric confidence intervals for regression-discontinuity designs. Technical report, Department of Economics, University of Michigan. Dehejia, R., and Wahba, S. (1999). Causal effects in non-experimental studies: re-evaluating the evaluation of training programs. Journal of the American Statistical Association 94, DiNardo, J., and D. Lee. (2010). Program evaluation and research designs. In Ashenfelter and Card (eds.), Handbook of Labor Economics, Vol. 4. Hahn, J., Todd, P., and Van Der Klaauw, W. (2001). Regression discontinuity, Econometrica 69, Imbens, G., and K. Kalyanaraman. (2012). Optimal bandwidth choice for the regression discontinuity estimator. Review of Economic Studies 79, Imbens, G., and Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics 142, Jacob, B., and Lefgren, L. (2004). Remedial education and student achievement: A regressiondiscontinuity design. Review of Economics and Statistics 86, LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs using experimental data. American Economic Review 76,

12 Lee, D. S., and Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature 48, Matsudaira, J, (2008). Mandatory summer school and student achievement. Journal of Econometrics 142, Porter, J. (2003). Estimation in the regression discontinuity model. Working Paper, Harvard University Department of Economics. Thistlewaite, D., and Campbell, D. (1960). Regression-discontinuity analysis: An alternative to the ex-post facto experiment. Journal of Educational Psychology 51, Van Der Klaauw, W. (2008). Regression-discontinuity analysis: A survey of recent developments in economics. Labour 22,

### Testing Stability of Regression Discontinuity Models

Testing Stability of Regression Discontinuity Models Giovanni Cerulli 1, Yingying Dong 2, Arthur Lewbel Ÿ3, and Alexander Poulsen 3 1 IRCrES-CNR, National Research Council of Italy 2 Department of Economics,

### Average Redistributional Effects. IFAI/IZA Conference on Labor Market Policy Evaluation

Average Redistributional Effects IFAI/IZA Conference on Labor Market Policy Evaluation Geert Ridder, Department of Economics, University of Southern California. October 10, 2006 1 Motivation Most papers

### Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

### Statistical Testing of Randomness Masaryk University in Brno Faculty of Informatics

Statistical Testing of Randomness Masaryk University in Brno Faculty of Informatics Jan Krhovják Basic Idea Behind the Statistical Tests Generated random sequences properties as sample drawn from uniform/rectangular

### Regression Discontinuity Marginal Threshold Treatment Effects

Regression Discontinuity Marginal Threshold Treatment Effects Yingying Dong and Arthur Lewbel California State University Fullerton and Boston College October 2010 Abstract In regression discontinuity

### Regression Discontinuity Design

Regression Discontinuity Design Marcelo Coca Perraillon University of Chicago April 24 & 29, 2013 1 / 40 Introduction Basics Method developed to estimate treatment effects in non-experimental settings

### Least Squares Estimation

Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David

### Inequality, Mobility and Income Distribution Comparisons

Fiscal Studies (1997) vol. 18, no. 3, pp. 93 30 Inequality, Mobility and Income Distribution Comparisons JOHN CREEDY * Abstract his paper examines the relationship between the cross-sectional and lifetime

### The Effect of Unemployment Benefits on the Duration of. Unemployment Insurance Receipt: New Evidence from a

The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013 David Card UC Berkeley, NBER and IZA Andrew Johnston

### Intro to Data Analysis, Economic Statistics and Econometrics

Intro to Data Analysis, Economic Statistics and Econometrics Statistics deals with the techniques for collecting and analyzing data that arise in many different contexts. Econometrics involves the development

### Online Appendices to the Corporate Propensity to Save

Online Appendices to the Corporate Propensity to Save Appendix A: Monte Carlo Experiments In order to allay skepticism of empirical results that have been produced by unusual estimators on fairly small

### The Effect of Health Insurance Coverage on the Reported Health of Young Adults

The Effect of Health Insurance Coverage on the Reported Health of Young Adults Eric Cardella Texas Tech University eric.cardella@ttu.edu Briggs Depew Louisiana State University bdepew@lsu.edu This Draft:

### CONTENTS OF DAY 2. II. Why Random Sampling is Important 9 A myth, an urban legend, and the real reason NOTES FOR SUMMER STATISTICS INSTITUTE COURSE

1 2 CONTENTS OF DAY 2 I. More Precise Definition of Simple Random Sample 3 Connection with independent random variables 3 Problems with small populations 8 II. Why Random Sampling is Important 9 A myth,

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### Good luck! BUSINESS STATISTICS FINAL EXAM INSTRUCTIONS. Name:

Glo bal Leadership M BA BUSINESS STATISTICS FINAL EXAM Name: INSTRUCTIONS 1. Do not open this exam until instructed to do so. 2. Be sure to fill in your name before starting the exam. 3. You have two hours

### Adequacy of Biomath. Models. Empirical Modeling Tools. Bayesian Modeling. Model Uncertainty / Selection

Directions in Statistical Methodology for Multivariable Predictive Modeling Frank E Harrell Jr University of Virginia Seattle WA 19May98 Overview of Modeling Process Model selection Regression shape Diagnostics

### Abstract Title Page Not included in page count.

Abstract Title Page Not included in page count. Title: Measuring Student Success from a Developmental Mathematics Course at an Elite Public Institution Authors and Affiliations: Julian Hsu, University

### arxiv: v1 [stat.ap] 18 Jun 2016

Estimating the Handicap Effect in the Go Game: A Regression Discontinuity Design Approach Kota Mori arxiv:1606.05778v1 [stat.ap] 18 Jun 2016 Abstract This paper provides an estimate for the handicap effect

### Interaction between quantitative predictors

Interaction between quantitative predictors In a first-order model like the ones we have discussed, the association between E(y) and a predictor x j does not depend on the value of the other predictors

### Minority Students. Stephen L. DesJardins. Center for the Study of Higher and Postsecondary Education. University of Michigan. and. Brian P.

The Impact of the Gates Millennium Scholars Program on the Retention, College Finance- and Work-Related Choices, and Future Educational Aspirations of Low-Income Minority Students Stephen L. DesJardins

### Means-tested complementary health insurance and healthcare utilisation in France: Evidence from a low-income population

Means-tested complementary health insurance and healthcare utilisation in France: Evidence from a low-income population Sophie Guthmullerᵃ1, Jérôme Wittwerᵃ ᵃ Université Paris-Dauphine, Paris, France Very

### Algebra 1 Course Information

Course Information Course Description: Students will study patterns, relations, and functions, and focus on the use of mathematical models to understand and analyze quantitative relationships. Through

### AN INTRODUCTION TO MATCHING METHODS FOR CAUSAL INFERENCE

AN INTRODUCTION TO MATCHING METHODS FOR CAUSAL INFERENCE AND THEIR IMPLEMENTATION IN STATA Barbara Sianesi IFS Stata Users Group Meeting Berlin, June 25, 2010 1 (PS)MATCHING IS EXTREMELY POPULAR 240,000

### Tutorial 5: Hypothesis Testing

Tutorial 5: Hypothesis Testing Rob Nicholls nicholls@mrc-lmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction................................ 1 2 Testing distributional assumptions....................

### Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

### Curso Avançado de Avaliação de Políticas Públicas e Projetos Sociais

Curso Avançado de Avaliação de Políticas Públicas e Projetos Sociais LIVROS i) Wooldridge "Econometric Analysis of Cross Section and Panel Data" MIT Press, 2002 ii) Angrist and Pischkle "Most Hamrmless

### Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

### IAPRI Quantitative Analysis Capacity Building Series. Multiple regression analysis & interpreting results

IAPRI Quantitative Analysis Capacity Building Series Multiple regression analysis & interpreting results How important is R-squared? R-squared Published in Agricultural Economics 0.45 Best article of the

### Estimation of σ 2, the variance of ɛ

Estimation of σ 2, the variance of ɛ The variance of the errors σ 2 indicates how much observations deviate from the fitted surface. If σ 2 is small, parameters β 0, β 1,..., β k will be reliably estimated

### A Regression Discontinuity Approach

Position Effects in Search Advertising: A Regression Discontinuity Approach Sridhar Narayanan Stanford University Kirthi Kalyanam Santa Clara University This version: January 2014 Abstract We investigate

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Gerry Hobbs, Department of Statistics, West Virginia University

Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

### Testing the Accuracy of Regression Discontinuity Analysis Using Experimental Benchmarks

Advance Access publication August 3, 2009 Political Analysis (2009) 17:400 417 doi:10.1093/pan/mpp018 Testing the Accuracy of Regression Discontinuity Analysis Using Experimental Benchmarks Donald P. Green

### From the help desk: Bootstrapped standard errors

The Stata Journal (2003) 3, Number 1, pp. 71 80 From the help desk: Bootstrapped standard errors Weihua Guan Stata Corporation Abstract. Bootstrapping is a nonparametric approach for evaluating the distribution

### Chapter 13 Introduction to Linear Regression and Correlation Analysis

Chapter 3 Student Lecture Notes 3- Chapter 3 Introduction to Linear Regression and Correlation Analsis Fall 2006 Fundamentals of Business Statistics Chapter Goals To understand the methods for displaing

### Polynomials. Dr. philippe B. laval Kennesaw State University. April 3, 2005

Polynomials Dr. philippe B. laval Kennesaw State University April 3, 2005 Abstract Handout on polynomials. The following topics are covered: Polynomial Functions End behavior Extrema Polynomial Division

### Introduction to Regression and Data Analysis

Statlab Workshop Introduction to Regression and Data Analysis with Dan Campbell and Sherlock Campbell October 28, 2008 I. The basics A. Types of variables Your variables may take several forms, and it

### e = random error, assumed to be normally distributed with mean 0 and standard deviation σ

1 Linear Regression 1.1 Simple Linear Regression Model The linear regression model is applied if we want to model a numeric response variable and its dependency on at least one numeric factor variable.

### Profit-sharing and the financial performance of firms: Evidence from Germany

Profit-sharing and the financial performance of firms: Evidence from Germany Kornelius Kraft and Marija Ugarković Department of Economics, University of Dortmund Vogelpothsweg 87, 44227 Dortmund, Germany,

### Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities

Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities Oscar Kipersztok Mathematics and Computing Technology Phantom Works, The Boeing Company P.O.Box 3707, MC: 7L-44 Seattle, WA 98124

### Moving the Goalposts: Addressing Limited Overlap in Estimation of Average Treatment Effects by Changing the Estimand

DISCUSSIO PAPER SERIES IZA DP o. 347 Moving the Goalposts: Addressing Limited Overlap in Estimation of Average Treatment Effects by Changing the Estimand Richard K. Crump V. Joseph Hotz Guido W. Imbens

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Testing for Granger causality between stock prices and economic growth

MPRA Munich Personal RePEc Archive Testing for Granger causality between stock prices and economic growth Pasquale Foresti 2006 Online at http://mpra.ub.uni-muenchen.de/2962/ MPRA Paper No. 2962, posted

### Some Polynomial Theorems. John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.

Some Polynomial Theorems by John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.com This paper contains a collection of 31 theorems, lemmas,

### Nonparametric statistics and model selection

Chapter 5 Nonparametric statistics and model selection In Chapter, we learned about the t-test and its variations. These were designed to compare sample means, and relied heavily on assumptions of normality.

### LOGISTIC REGRESSION. Nitin R Patel. where the dependent variable, y, is binary (for convenience we often code these values as

LOGISTIC REGRESSION Nitin R Patel Logistic regression extends the ideas of multiple linear regression to the situation where the dependent variable, y, is binary (for convenience we often code these values

### Chapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 )

Chapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 ) and Neural Networks( 類 神 經 網 路 ) 許 湘 伶 Applied Linear Regression Models (Kutner, Nachtsheim, Neter, Li) hsuhl (NUK) LR Chap 10 1 / 35 13 Examples

### Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

### Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive Amounts

### Factors affecting online sales

Factors affecting online sales Table of contents Summary... 1 Research questions... 1 The dataset... 2 Descriptive statistics: The exploratory stage... 3 Confidence intervals... 4 Hypothesis tests... 4

### Marginal Person. Average Person. (Average Return of College Goers) Return, Cost. (Average Return in the Population) (Marginal Return)

1 2 3 Marginal Person Average Person (Average Return of College Goers) Return, Cost (Average Return in the Population) 4 (Marginal Return) 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

### 11. Analysis of Case-control Studies Logistic Regression

Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

### Basic Statistics and Data Analysis for Health Researchers from Foreign Countries

Basic Statistics and Data Analysis for Health Researchers from Foreign Countries Volkert Siersma siersma@sund.ku.dk The Research Unit for General Practice in Copenhagen Dias 1 Content Quantifying association

### Regression. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.

Class: Date: Regression Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Given the least squares regression line y8 = 5 2x: a. the relationship between

### Nonlinear Regression Functions. SW Ch 8 1/54/

Nonlinear Regression Functions SW Ch 8 1/54/ The TestScore STR relation looks linear (maybe) SW Ch 8 2/54/ But the TestScore Income relation looks nonlinear... SW Ch 8 3/54/ Nonlinear Regression General

### Numerical Summarization of Data OPRE 6301

Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

### Purpose of Time Series Analysis. Autocovariance Function. Autocorrelation Function. Part 3: Time Series I

Part 3: Time Series I Purpose of Time Series Analysis (Figure from Panofsky and Brier 1968) Autocorrelation Function Harmonic Analysis Spectrum Analysis Data Window Significance Tests Some major purposes

### Bootstrapping Big Data

Bootstrapping Big Data Ariel Kleiner Ameet Talwalkar Purnamrita Sarkar Michael I. Jordan Computer Science Division University of California, Berkeley {akleiner, ameet, psarkar, jordan}@eecs.berkeley.edu

### Regression Analysis: Basic Concepts

The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance

### On Marginal Effects in Semiparametric Censored Regression Models

On Marginal Effects in Semiparametric Censored Regression Models Bo E. Honoré September 3, 2008 Introduction It is often argued that estimation of semiparametric censored regression models such as the

### ESTIMATING THE EFFECT OF FINANCIAL AID OFFERS ON COLLEGE ENROLLMENT: A REGRESSION DISCONTINUITY APPROACH

INTERNATIONAL ECONOMIC REVIEW Vol. 43, No. 4, November 2002 ESTIMATING THE EFFECT OF FINANCIAL AID OFFERS ON COLLEGE ENROLLMENT: A REGRESSION DISCONTINUITY APPROACH BY WILBERT VAN DER KLAAUW 1 Department

### Trend and Seasonal Components

Chapter 2 Trend and Seasonal Components If the plot of a TS reveals an increase of the seasonal and noise fluctuations with the level of the process then some transformation may be necessary before doing

### Discussion of Momentum and Autocorrelation in Stock Returns

Discussion of Momentum and Autocorrelation in Stock Returns Joseph Chen University of Southern California Harrison Hong Stanford University Jegadeesh and Titman (1993) document individual stock momentum:

### How To Run Statistical Tests in Excel

How To Run Statistical Tests in Excel Microsoft Excel is your best tool for storing and manipulating data, calculating basic descriptive statistics such as means and standard deviations, and conducting

### Using Excel for inferential statistics

FACT SHEET Using Excel for inferential statistics Introduction When you collect data, you expect a certain amount of variation, just caused by chance. A wide variety of statistical tests can be applied

### NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

### Multivariate Analysis of Ecological Data

Multivariate Analysis of Ecological Data MICHAEL GREENACRE Professor of Statistics at the Pompeu Fabra University in Barcelona, Spain RAUL PRIMICERIO Associate Professor of Ecology, Evolutionary Biology

### South Carolina College- and Career-Ready (SCCCR) Probability and Statistics

South Carolina College- and Career-Ready (SCCCR) Probability and Statistics South Carolina College- and Career-Ready Mathematical Process Standards The South Carolina College- and Career-Ready (SCCCR)

### Forecast. Forecast is the linear function with estimated coefficients. Compute with predict command

Forecast Forecast is the linear function with estimated coefficients T T + h = b0 + b1timet + h Compute with predict command Compute residuals Forecast Intervals eˆ t = = y y t+ h t+ h yˆ b t+ h 0 b Time

### The Effect of Health Insurance Coverage on the Use of Medical Services * March 8, 2009. Abstract

The Effect of Health Insurance Coverage on the Use of Medical Services * Michael Anderson UC Berkeley mlanderson@berkeley.edu Carlos Dobkin UC Santa Cruz cdobkin@ucsc.edu March 8, 2009 Abstract Despite

### Testing against a Change from Short to Long Memory

Testing against a Change from Short to Long Memory Uwe Hassler and Jan Scheithauer Goethe-University Frankfurt This version: December 9, 2007 Abstract This paper studies some well-known tests for the null

### 1. The parameters to be estimated in the simple linear regression model Y=α+βx+ε ε~n(0,σ) are: a) α, β, σ b) α, β, ε c) a, b, s d) ε, 0, σ

STA 3024 Practice Problems Exam 2 NOTE: These are just Practice Problems. This is NOT meant to look just like the test, and it is NOT the only thing that you should study. Make sure you know all the material

### STRATEGIC CAPACITY PLANNING USING STOCK CONTROL MODEL

Session 6. Applications of Mathematical Methods to Logistics and Business Proceedings of the 9th International Conference Reliability and Statistics in Transportation and Communication (RelStat 09), 21

### Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

### DATA INTERPRETATION AND STATISTICS

PholC60 September 001 DATA INTERPRETATION AND STATISTICS Books A easy and systematic introductory text is Essentials of Medical Statistics by Betty Kirkwood, published by Blackwell at about 14. DESCRIPTIVE

### Statistical Analysis on Relation between Workers Information Security Awareness and the Behaviors in Japan

Statistical Analysis on Relation between Workers Information Security Awareness and the Behaviors in Japan Toshihiko Takemura Kansai University This paper discusses the relationship between information

### Elements of statistics (MATH0487-1)

Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium December 10, 2012 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis -

### Permutation Tests for Comparing Two Populations

Permutation Tests for Comparing Two Populations Ferry Butar Butar, Ph.D. Jae-Wan Park Abstract Permutation tests for comparing two populations could be widely used in practice because of flexibility of

### From the help desk: Swamy s random-coefficients model

The Stata Journal (2003) 3, Number 3, pp. 302 308 From the help desk: Swamy s random-coefficients model Brian P. Poi Stata Corporation Abstract. This article discusses the Swamy (1970) random-coefficients

### Stochastic Inventory Control

Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the

### 2013 MBA Jump Start Program. Statistics Module Part 3

2013 MBA Jump Start Program Module 1: Statistics Thomas Gilbert Part 3 Statistics Module Part 3 Hypothesis Testing (Inference) Regressions 2 1 Making an Investment Decision A researcher in your firm just

### Interpretation of Somers D under four simple models

Interpretation of Somers D under four simple models Roger B. Newson 03 September, 04 Introduction Somers D is an ordinal measure of association introduced by Somers (96)[9]. It can be defined in terms

### Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Name: Section: I pledge my honor that I have not violated the Honor Code Signature: This exam has 34 pages. You have 3 hours to complete this

### Difference in differences and Regression Discontinuity Design

Difference in differences and Regression Discontinuity Design Majeure Economie September 2011 1 Difference in differences Intuition Identification of a causal effect Discussion of the assumption Examples

### Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario

Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario I have found that the best way to practice regression is by brute force That is, given nothing but a dataset and your mind, compute everything

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### Appendix 1: Time series analysis of peak-rate years and synchrony testing.

Appendix 1: Time series analysis of peak-rate years and synchrony testing. Overview The raw data are accessible at Figshare ( Time series of global resources, DOI 10.6084/m9.figshare.929619), sources are

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

### Testing against a Change from Short to Long Memory

Testing against a Change from Short to Long Memory Uwe Hassler and Jan Scheithauer Goethe-University Frankfurt This version: January 2, 2008 Abstract This paper studies some well-known tests for the null

### Discussion Section 4 ECON 139/239 2010 Summer Term II

Discussion Section 4 ECON 139/239 2010 Summer Term II 1. Let s use the CollegeDistance.csv data again. (a) An education advocacy group argues that, on average, a person s educational attainment would increase

### The Impact of Washington State Achievers Scholarship on Student Outcomes. Stephen L. DesJardins

The Impact of Washington State Achievers Scholarship on Student Outcomes Stephen L. DesJardins Center for the Study of Higher and Postsecondary Education University of Michigan and Brian P. McCall Center

### Analysis of Bayesian Dynamic Linear Models

Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main

### Multiple Imputation for Missing Data: A Cautionary Tale

Multiple Imputation for Missing Data: A Cautionary Tale Paul D. Allison University of Pennsylvania Address correspondence to Paul D. Allison, Sociology Department, University of Pennsylvania, 3718 Locust

### The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach. Alan S. Gerber, Daniel P. Kessler, Marc Meredith

The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach Alan S. Gerber, Daniel P. Kessler, Marc Meredith Alan S. Gerber is George C. and Dorathea S. Dilley Professor of Political

### Note 2 to Computer class: Standard mis-specification tests

Note 2 to Computer class: Standard mis-specification tests Ragnar Nymoen September 2, 2013 1 Why mis-specification testing of econometric models? As econometricians we must relate to the fact that the

### PCHS ALGEBRA PLACEMENT TEST

MATHEMATICS Students must pass all math courses with a C or better to advance to the next math level. Only classes passed with a C or better will count towards meeting college entrance requirements. If

### Basic Concepts in Research and Data Analysis

Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the