# Life Table Analysis using Weighted Survey Data

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Life Table Analysis using Weighted Survey Data James G. Booth and Thomas A. Hirschl June 2005 Abstract Formulas for constructing valid pointwise confidence bands for survival distributions, estimated using unequal probability samples, are derived. In addition, a test statistic for comparing estimated survival distributions from several groups is proposed. The methodology can be applied when survival times are subject to an independent censoring mechanism. The methods are illustrated using data from the Panel Survey on Income Dynamics on time until first home purchase, broken down by race and education level. Introduction Survey data is often collected, either intentionally or unintentionally, using an unequal probability sampling scheme. Individuals with certain characteristics are more or less likely to be included in the sample than others. In practice the selection probabilities for various subgroups may be unknown in advance, but may be estimated after the fact; for example, by calibrating post-stratification rates to known population proportions from the complete census. Information about the unequal selection probabilities is often provided to the public in the form of Department of Biological Statistics and Computational Biology, Warren Hall, Cornell University, NY Department of Development Sociology, Cornell University

2 sampling weights which must be taken into account when estimating marginal properties of the population. Thoughout this paper we assume the following, idealized, probability sampling model. Suppose that the population of interest is divided into S strata, with a proportion π s coming from stratum s. Individuals are selected by first choosing a stratum using probabilities τ,..., τ S, and then selecting an individual at random from the chosen stratum. We suppose that the population is large enough that the difference between with and without replacement sampling is negligible. The sampling weights associated with individuals from the different strata are then proportional to the ratios, π s /τ s, g =,..., S. As noted above the selection probabilities must be taken into account when estimating marginal characteristics. For example, suppose we want to estimate the proportion of the population to whom a certain event has happened. Let E i be an event indicator associated with the ith sampled individual. Then, the population characteristic of interest is P = P r(e i = ). It is straightforward to verify that an unbiased estimate of P based on a sample of size n is ˆP = n π si E i, () n τ si i= where s i is the stratum from which the ith individual is selected. This is, in fact, the infinite population version of the famous Horwitz-Thompson estimator (see e.g. Cochran, 977, Section 9.A.7), and is consistent for P as the sample size increases. If the probability ratios in () are unknown, but proportional sampling weights, w i π si /τ si, i =,..., n, are available for sampled individuals, a consistent estimate of P is obtained as a ratio of two means, n n i= ˆP = w ie i n n i= w. i In this paper we develop methods for incorporating sampling weights into the analysis of discrete time-to-event data. Specifically, in Section 2 we modify the 2

3 standard survival function estimate based on a life table to account for individual sampling weights. In Section 3 we use the delta method to derive standard error formulas which may be used to construst pointwise confidence bands for the survival function, and in Section 4 we propose a test-statistic for comparing several survival functions based on data from different groups. We illustrate the proposed methodology in Section 5 using data on homeownership taken from the Panel Study of Income Dynamics (Hill, 992). We conclude with some discussion of the appropriateness of our sampling model in practice. 2 Application to Life Table Analysis In longitudinal surveys it is often of interest to estimate a survival curve for the population. What proportion of the population survive beyond the jth time interval without a particular event happening? This probability can be represented as Q j = q q 2 q j, (2) where q r is the probability of surviving the rth interval, given that you survived until the end of the previous one. The distribution function for the time-to-event, corresponding to (2), is given by P j = Q j, j =, 2,.... Let R ij be the indicator that the the i individual is at risk at the beginning of interval j, and let E ij be the indicator that the event occurs in interval j. Let R j denote the set of individuals at risk (the risk set) at the beginning of interval j. Then, q j = p j, where p j = P r(e ij = i R j ) = P r(e ij =, i R j ) P r(i R j ) It follows from () that a consistent estimate of p j is given by ˆp j = n π si n i= n i= τ si E ij π si = τ si R ij 3 = P r(e ij = ) P r(i R j ). (3) n i= w ie ij i= w. (4) ir ij n

4 Clearly, the proportionality constant associated with the sampling weights cancels out in the calculation of ˆp j in (4). Note that ˆp j is a ratio of two sample means, ˆµ E j and ˆµ R j, say, that are unbiased estimates of their corresponding population values µ E j and µ R j. Also, note that ˆq j = i= w i(r ij E ij ) i= w ir ij = i= w ir i,j+ i= w ir ij implies ˆQ j = i= w ir i,j+ i= w i. Thus, in the unweighted (unit weight) case, ˆQj is simply the proportion of individuals who survive until the beginning of the (j + )st interval. If some individuals are censored or withdraw during the jth interval, the estimate (4) will tend to underestimate p j. Under the assumption that censoring times are independent of the times to the event of interest, we propose the following adjustment to ˆp j, where R ij ˆp j = n i= w ie ij n i= w ir ij = ˆµE j, ˆµ R j = R ij C ij /2, and C ij is a censoring/withdrawal indicator for the ith individual in interval j. This reduces to the standard actuarial adjustment in the unweighted case (Kalbfleisch & Prentice, 980, Section.3). As in the unweighted case, ˆp j is asymptotically biased as an estimator of p j. However, as noted by Lawless (2003, Section 3.6.2), the standard life table estimates are acceptable under random independent censorship provided that censoring is fairly evenly distributed across individual intervals and not too heavy. The survival function estimator discussed in this section can be computed using standard software by creating integer sampling weights, and treating them as observed frequencies (for example, using the frequency option in the SAS (2003) 4

5 lifetest procedure). This can be accomplished in practice by multiplying the sampling weights provided by a large positive constant and then rounding to the nearest integer. The problem with this approach is that it artificially inflates the sample size, with the result that standard errors and test statistics produced by the software are invalid. 3 Standard Errors and Confidence Bands Since the estimate of Q j is a product of ratios of means, standard errors for constructing confidence bands for the survival function can be estimated using the delta method (Stuart & Ord, 994, Section 0.5-7). However, the application of the delta method with weighted survival data is slighty different from the unweighted case described in standard texts such as Kalbfleisch & Prentice (980) or Lawless (2003). Specifically, since ˆp j is a ratio of two means, an exact variance formula is not available. In contrast, in the unweighted case ˆp j is (conditionally) a binomial proportion whose variance has a known form. As in the unweighted case, the estimators, ˆp,..., ˆp j, are asymptotically uncorrelated. To see this, define H i = {(w i E ij, w i R ij, w i C ij ), j =, 2,...}, and notice that H,..., H n are i.i.d.. Also, define H(j) = {(i, H i ), i / R j }. Thus, H(j) represents the history of the sampled individuals up until the beginning of the jth interval. Now, the linear approximation, ˆp j p j + µ R j (ˆµ E j p j ˆµ R j ) = p j + nµ R j n w i (E ij p j R ij ), (5) i= 5

6 implies, for j < k, cov(ˆp j, ˆp k ) = = = 0. nµ R j µr k nµ R j µr k nµ R j µr k E {w i (E ij p j R ij )w i (E ik p k R ik )} E [E {w i (E ij p j R ij )w i (E ik p k R ik ) H(k)}] E [w i (E ij p j R ij ) E {w i (E ik p k R ik ) H(k)}] (Technically, this argument shows that cov(ˆp j, ˆp k ) = o(/n).) The same argument works when there is censoring, with R replaced by R, and p j replaced by p j, the asymptotic limit of ˆp j. The approximation (5) leads to a variance formula for ˆp j given by and its estimator var(ˆp j ) var(ˆp j ) n(µ R j )2 E {w i(e ij p j R ij )} 2, (6) (nˆµ R j )2 n {w i (E ij ˆp j R ij )} 2. (7) i= To obtain a variance formula for the survival probabilities we use the linear approximation ˆQ j = j ( ˆp i ) Q j i= ( j j ) q k (ˆp i p i ). i= Hence, since the ˆp i s are asymptotically uncorrelated, the delta method implies k i the approximate variance formula, ( j j ) 2 var( ˆQ j ) q k var(ˆp i ) = Q 2 j i= k i j i= var(ˆp i ) q 2 i. (8) An asymptotic confidence interval for Q j based on using (8) directly is ˆQ j ± z ŝe( ˆQ j ), where se denotes standard error, and z is a quantile of the standard 6

7 normal distribution. It is well known that this direct method can yield nonsensical limits, outside of the unit interval, especially when ˆQ j is close to zero or one. Hence, it is standard practice (Kalbfleisch & Prentice, 980, p.4) to calculate the interval for the transformed parameter V j = log( log Q j ), which has an unrestricted range, and then transform back. This results in confidence limits for Q j given by ˆQ exp[±z c se( ˆV j )] j, where se( ˆV j ) = se( ˆQ j )/ Q j log Q j. Even with this modification it is possible that the confidence bands for the survival function may not be monotone, especially in the tail of the distribution where data may be scarce. Thus, we also suggest following the standard practice of enforcing monotonicity (Anderson et al., 992, pp ). 4 Comparison of Distribution Functions Suppose the subjects in the sample can be separated into G groups, for example, based on years of education. In this section we propose a test statistic for contrasting the distribution function estimates from the different groups. We use the same notation introduced in the previous section, except that an additional subscript g is added to identify the group. Thus, for example, p gj is the probability that the event of interest occurs during the jth time interval for individuals in group g, conditional on their being at risk a the beginning of the interval. An estimate of this probability is given by the ratio ˆµ E gj/ˆµ R gj (or ˆµ E gj/ˆµ R gj, if there is some censoring). The combined estimate of the event occuring in time interval j, based on the data from all the groups, is given by ˆp j = G g= n g ˆµ E gj G g= n g ˆµ R gj. (9) 7

8 A measure of the discrepancy between the distribution in a particular group and the combined distribution is given by the quantity g = j n g (ˆµ E gj ˆp j ˆµ R gj). (0) Note that (9) implies that g g = 0. Let = (,..., G ), and let Σ denote the estimate of the variance-covariance matrix of, obtained using the delta method. (An asymptotic formula for Σ is given in the Appendix.) Then, a test of the hypothesis of no difference between groups (homogeneity) is obtained by comparing the statistic, χ 2 = ˆΣ, () where Σ is a generalized inverse of Σ, to a chi-squared distribution with G degrees of freedom. 5 Application to Homeownership The Panel Study of Income Dynamics (PSID) is a nationally representative, longitudinal sample of households and families interviewed since 968 (Hill, 992). It constitutes the longest running panel data set in the United States, and was specifically designed to track social dynamics over time. Since the PSID employs an unequal probability sampling scheme, each sampled individual is assigned a weight to correct for sampling bias. In this section we use a subset of the PSID database to illustrate the methods described in this paper. Specifically, we focus on the distribution of the number of years to first home ownership among individuals who were renters at age 25, and how this distribution depends upon race (Black or White) and education (< 2, 2, or > 2 years). Summary statistics for the sampling weights in the six race/education categories are provided in Table. The overall sample sizes clearly 8

9 indicate that blacks were oversampled, and this is reflected in their lower average sampling weights. Also note that there is considerable amount of variability in the sampling weights within each race/education category. Education Level < 2 2 > 2 Overall Blacks Sample size Mean weight Std.Dev Whites Sample size Mean weight Std.Dev Table : Summary statistics for individual sampling weights by race and years of education. The upper panels in Figure show the estimated distributions of time to first homeownership among whites and blacks along with 95% pointwise confidence bands. The distributions are clearly different, with blacks tending to take longer to purchase their first home. For example, about 80% of whites have been homeowners by age 35, compared with about 50% of blacks. The confidence bands are wider for the black distribution, in part because of the smaller sample size, but also because of the higher censoring rate (088 blacks censored compared with 586 whites). The lower panels in Figure show the distribution function estimates broken down by education. Among whites, the distributions appear to be very consistent across education levels. The chisquared statistic discussed in Section 4 is 3. with 2 degrees of freedom in this case, indicating statistically significant differences among the education categories. This is not surprising given the relatively large sample sizes, and consequential high power of the test. There are much 9

10 larger differences in the distributions among blacks by education, resulting in a larger chisquared statistic of 9.9 with 2 degrees of freedom. Here the distributions appear to be stochastically ordered with the probability of homeownership increasing with education level, particularly in the 35 to 45 age range. 6 Discussion We have proposed methods for utilizing sampling weights when analyzing timeto-event data, based on a simple unequal probability sampling model. While this model may not exactly reflect the sampling process, we contend that it serves as a reasonable probabilistic approximation under a variety of realistic sampling schemes. It is clear, for example, that the methods are valid for a stratified random sample. Suppose there are known to be N s members of stratum s, and it was possible to identify members of the various strata in advance. If a predetermined number, n s, of individuals from statum s are selected, then the estimate of ˆp j in (4), and its asymptotic variance given in (6), remain unchanged with π s and τ s replaced by the corresponding proportions, N s /N and n s /n. In practice, separate sampling frames for the different strata will typically not be available. A more realistic sampling scheme might involve taking in k d samples (of say households) in a collection of districts or regions, d =,..., D. The value of k d may be varied in order to oversample certain groups based on known demographics for the different districts. With this scheme the sample sizes, n s, are unknown in advance, and so the marginal selection probabilities, τ s, s =,..., S, must be estimated based on post-stratification rates. Methods that account for the estimation of the sampling weights would be considerably more complicated than those discussed in this paper. It is not clear to what extent such an extension is important, but it is an interesting topic worthy of further study. 0

11 Appendix An asymptotic formula for the variance-covariance matrix, Σ, defined in Section 4 can be obtained using the delta method. Note that g = j gj, where and that gj = n g (ˆµ E gj ˆp j ˆµ R gj) n g (ˆµ E gj p j ˆµ R gj ˆp j µ R gj) + constant, It follows that, gj n g { ˆp j = ˆµE j ˆµ R j p j + µ R j = p j + µ R j (ˆµ E gj p j ˆµ R gj) µr gj µ R j (ˆµ E j p j ˆµ R j ) + G g= G k= The arguments in Section 3 imply that µ R j (µ E j p j µ R j ) n g n (ˆµE gj p j ˆµ R gj) + constant. n k n (ˆµE kj p j ˆµ R kj) cov(ˆµ E gj p j ˆµ R gj, ˆµ E gk p k ˆµ R gk) 0. } + constant. (A.) Hence, equation A. implies var( g ) j var( gj) and cov( g, h ) j cov( gj, hj ), where and ( var( gj ) = n 2 g + n gµ R gj nµ R j ( µ R gj cov( gj, hj ) = n g n h { µ R gj µ R hj (µ R j )2 n gµ R gj nµ R j µ R j ) 2 var(ˆµ E gj p j ˆµ R gj) ) 2 G k= k g ( nk ) 2 var(ˆµ E kj p j ˆµ R n kj), ( nk ) 2 var(ˆµ E kj p j ˆµ R n kj) var(ˆµ E gj p j ˆµ R gj) n hµ R hj var(ˆµ E nµ R hj p j ˆµ R hj) j }.

12 References ANDERSON, P., BORGAN, O., GILL, R. & KIEDING, N. (992). Models Based on Counting Processes. Springer. Statistical COCHRAN, W. G. (977). Sampling Techniques. Wiley, 3rd ed. HILL, M. S. (992). The Panel Study of Income Dynamics: A Users Guide. Newbury Park, CA: Sage Publications. KALBFLEISCH, J. D. & PRENTICE, R. L. (980). The Statistical Analysis of Failure Time Data. Wiley. LAWLESS, J. F. (2003). Statistical Models and Methods for Lifetime Data. Wiley, 2nd ed. SAS (2003). Lifetest procedure. Cary, NC: SAS Institute Inc. STUART, A. & ORD, K. (994). Kendall s Advanced Theory of Statistics, vol.. Edward Arnold, 6th ed. 2

13 Distribution of years to first homeownership Whites Combined Blacks Combined YEARS Whites by Education PROBABILITY YEARS Blacks by Education PROBABILITY Figure : Upper panels show estimated distribution functions with pointwise 95% confidence limits. Lower panels show estimates for three education categories: highschool dropout (black), highschool graduate (red), and some college (blue) 3

### Survey, Statistics and Psychometrics Core Research Facility University of Nebraska-Lincoln. Log-Rank Test for More Than Two Groups

Survey, Statistics and Psychometrics Core Research Facility University of Nebraska-Lincoln Log-Rank Test for More Than Two Groups Prepared by Harlan Sayles (SRAM) Revised by Julia Soulakova (Statistics)

### INTRODUCTION TO SURVEY DATA ANALYSIS THROUGH STATISTICAL PACKAGES

INTRODUCTION TO SURVEY DATA ANALYSIS THROUGH STATISTICAL PACKAGES Hukum Chandra Indian Agricultural Statistics Research Institute, New Delhi-110012 1. INTRODUCTION A sample survey is a process for collecting

### Using Repeated Measures Techniques To Analyze Cluster-correlated Survey Responses

Using Repeated Measures Techniques To Analyze Cluster-correlated Survey Responses G. Gordon Brown, Celia R. Eicheldinger, and James R. Chromy RTI International, Research Triangle Park, NC 27709 Abstract

### SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

### Confidence Intervals in Public Health

Confidence Intervals in Public Health When public health practitioners use health statistics, sometimes they are interested in the actual number of health events, but more often they use the statistics

### A C T R esearcli R e p o rt S eries 2 0 0 5. Using ACT Assessment Scores to Set Benchmarks for College Readiness. IJeff Allen.

A C T R esearcli R e p o rt S eries 2 0 0 5 Using ACT Assessment Scores to Set Benchmarks for College Readiness IJeff Allen Jim Sconing ACT August 2005 For additional copies write: ACT Research Report

### Lecture 2 ESTIMATING THE SURVIVAL FUNCTION. One-sample nonparametric methods

Lecture 2 ESTIMATING THE SURVIVAL FUNCTION One-sample nonparametric methods There are commonly three methods for estimating a survivorship function S(t) = P (T > t) without resorting to parametric models:

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Least Squares Estimation

Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David

### 9.2 Summation Notation

9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a

### Simple Random Sampling

Source: Frerichs, R.R. Rapid Surveys (unpublished), 2008. NOT FOR COMMERCIAL DISTRIBUTION 3 Simple Random Sampling 3.1 INTRODUCTION Everyone mentions simple random sampling, but few use this method for

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Poisson Models for Count Data

Chapter 4 Poisson Models for Count Data In this chapter we study log-linear models for count data under the assumption of a Poisson error structure. These models have many applications, not only to the

### Variability in Life-Table Estimates Based on the National Health Interview Survey Linked Mortality Files

Variability in Life-Table Estimates Based on the National Health Interview Survey Linked Mortality Files Van Parsons 1, Nathaniel Schenker, Kimberly Lochner, Gloria Wheatcroft and Elsie Pamuk 1 National

### MINITAB ASSISTANT WHITE PAPER

MINITAB ASSISTANT WHITE PAPER This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. One-Way

### Multivariate Normal Distribution

Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

### The Delta Method and Applications

Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and

### A composite estimator for stratified two stage cluster sampling

Communications for Statistical Applications and Methods 2016, Vol. 23, No. 1, 47 http://dx.doi.org/.31/csam.2016.23.1.047 Print ISSN 2287-7843 / Online ISSN 2383-477 A composite estimator for stratified

### SYSTEMS OF REGRESSION EQUATIONS

SYSTEMS OF REGRESSION EQUATIONS 1. MULTIPLE EQUATIONS y nt = x nt n + u nt, n = 1,...,N, t = 1,...,T, x nt is 1 k, and n is k 1. This is a version of the standard regression model where the observations

### PROBABILITY AND STATISTICS. Ma 527. 1. To teach a knowledge of combinatorial reasoning.

PROBABILITY AND STATISTICS Ma 527 Course Description Prefaced by a study of the foundations of probability and statistics, this course is an extension of the elements of probability and statistics introduced

### Chapter 6: Point Estimation. Fall 2011. - Probability & Statistics

STAT355 Chapter 6: Point Estimation Fall 2011 Chapter Fall 2011 6: Point1 Estimat / 18 Chap 6 - Point Estimation 1 6.1 Some general Concepts of Point Estimation Point Estimate Unbiasedness Principle of

### It is important to bear in mind that one of the first three subscripts is redundant since k = i -j +3.

IDENTIFICATION AND ESTIMATION OF AGE, PERIOD AND COHORT EFFECTS IN THE ANALYSIS OF DISCRETE ARCHIVAL DATA Stephen E. Fienberg, University of Minnesota William M. Mason, University of Michigan 1. INTRODUCTION

### Survey Inference for Subpopulations

American Journal of Epidemiology Vol. 144, No. 1 Printed In U.S.A Survey Inference for Subpopulations Barry I. Graubard 1 and Edward. Korn 2 One frequently analyzes a subset of the data collected in a

### WHERE DOES THE 10% CONDITION COME FROM?

1 WHERE DOES THE 10% CONDITION COME FROM? The text has mentioned The 10% Condition (at least) twice so far: p. 407 Bernoulli trials must be independent. If that assumption is violated, it is still okay

### An introduction to Value-at-Risk Learning Curve September 2003

An introduction to Value-at-Risk Learning Curve September 2003 Value-at-Risk The introduction of Value-at-Risk (VaR) as an accepted methodology for quantifying market risk is part of the evolution of risk

### 171:290 Model Selection Lecture II: The Akaike Information Criterion

171:290 Model Selection Lecture II: The Akaike Information Criterion Department of Biostatistics Department of Statistics and Actuarial Science August 28, 2012 Introduction AIC, the Akaike Information

### Simple Second Order Chi-Square Correction

Simple Second Order Chi-Square Correction Tihomir Asparouhov and Bengt Muthén May 3, 2010 1 1 Introduction In this note we describe the second order correction for the chi-square statistic implemented

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Statistical Models in R

Statistical Models in R Some Examples Steven Buechler Department of Mathematics 276B Hurley Hall; 1-6233 Fall, 2007 Outline Statistical Models Structure of models in R Model Assessment (Part IA) Anova

### Review. March 21, 2011. 155S7.1 2_3 Estimating a Population Proportion. Chapter 7 Estimates and Sample Sizes. Test 2 (Chapters 4, 5, & 6) Results

MAT 155 Statistical Analysis Dr. Claude Moore Cape Fear Community College Chapter 7 Estimates and Sample Sizes 7 1 Review and Preview 7 2 Estimating a Population Proportion 7 3 Estimating a Population

### Annex 6 BEST PRACTICE EXAMPLES FOCUSING ON SAMPLE SIZE AND RELIABILITY CALCULATIONS AND SAMPLING FOR VALIDATION/VERIFICATION. (Version 01.

Page 1 BEST PRACTICE EXAMPLES FOCUSING ON SAMPLE SIZE AND RELIABILITY CALCULATIONS AND SAMPLING FOR VALIDATION/VERIFICATION (Version 01.1) I. Introduction 1. The clean development mechanism (CDM) Executive

### Adjusting the Mantel Haenszel Test Statistic and Odds Ratio for Cluster Sampling

Adjusting the Mantel Haenszel Test Statistic and Odds Ratio for Cluster Sampling Gilles Lamothe Department of Mathematics and Statistics, University of Ottawa October 15, 2011 1 Introduction The purpose

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### Power and sample size in multilevel modeling

Snijders, Tom A.B. Power and Sample Size in Multilevel Linear Models. In: B.S. Everitt and D.C. Howell (eds.), Encyclopedia of Statistics in Behavioral Science. Volume 3, 1570 1573. Chicester (etc.): Wiley,

### Department of Economics

Department of Economics On Testing for Diagonality of Large Dimensional Covariance Matrices George Kapetanios Working Paper No. 526 October 2004 ISSN 1473-0278 On Testing for Diagonality of Large Dimensional

### 2DI36 Statistics. 2DI36 Part II (Chapter 7 of MR)

2DI36 Statistics 2DI36 Part II (Chapter 7 of MR) What Have we Done so Far? Last time we introduced the concept of a dataset and seen how we can represent it in various ways But, how did this dataset came

### 4. Continuous Random Variables, the Pareto and Normal Distributions

4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random

### The Variability of P-Values. Summary

The Variability of P-Values Dennis D. Boos Department of Statistics North Carolina State University Raleigh, NC 27695-8203 boos@stat.ncsu.edu August 15, 2009 NC State Statistics Departement Tech Report

### The SURVEYFREQ Procedure in SAS 9.2: Avoiding FREQuent Mistakes When Analyzing Survey Data ABSTRACT INTRODUCTION SURVEY DESIGN 101 WHY STRATIFY?

The SURVEYFREQ Procedure in SAS 9.2: Avoiding FREQuent Mistakes When Analyzing Survey Data Kathryn Martin, Maternal, Child and Adolescent Health Program, California Department of Public Health, ABSTRACT

### IBM SPSS Complex Samples 22

IBM SPSS Complex Samples 22 Note Before using this information and the product it supports, read the information in Notices on page 51. Product Information This edition applies to version 22, release 0,

### Fitting Subject-specific Curves to Grouped Longitudinal Data

Fitting Subject-specific Curves to Grouped Longitudinal Data Djeundje, Viani Heriot-Watt University, Department of Actuarial Mathematics & Statistics Edinburgh, EH14 4AS, UK E-mail: vad5@hw.ac.uk Currie,

### Multimode Strategies for Designing Establishment Surveys

Multimode Strategies for Designing Establishment Surveys Shelton M. Jones RTI International, 3 Cornwallis Rd., Research Triangle Park, NC, 2779 Abstract Design strategies are discussed that improve participation

### Master s Theory Exam Spring 2006

Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

### A POPULATION MEAN, CONFIDENCE INTERVALS AND HYPOTHESIS TESTING

CHAPTER 5. A POPULATION MEAN, CONFIDENCE INTERVALS AND HYPOTHESIS TESTING 5.1 Concepts When a number of animals or plots are exposed to a certain treatment, we usually estimate the effect of the treatment

### A Model of Optimum Tariff in Vehicle Fleet Insurance

A Model of Optimum Tariff in Vehicle Fleet Insurance. Bouhetala and F.Belhia and R.Salmi Statistics and Probability Department Bp, 3, El-Alia, USTHB, Bab-Ezzouar, Alger Algeria. Summary: An approach about

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Elementary Statistics

Elementary Statistics Chapter 1 Dr. Ghamsary Page 1 Elementary Statistics M. Ghamsary, Ph.D. Chap 01 1 Elementary Statistics Chapter 1 Dr. Ghamsary Page 2 Statistics: Statistics is the science of collecting,

### Multivariate Analysis of Variance (MANOVA): I. Theory

Gregory Carey, 1998 MANOVA: I - 1 Multivariate Analysis of Variance (MANOVA): I. Theory Introduction The purpose of a t test is to assess the likelihood that the means for two groups are sampled from the

### Accounting for complex survey design in modeling usual intake. Kevin W. Dodd, PhD National Cancer Institute

Accounting for complex survey design in modeling usual intake Kevin W. Dodd, PhD National Cancer Institute Slide 1 Hello and welcome to today s webinar, the fourth in the Measurement Error Webinar Series.

### LOGNORMAL MODEL FOR STOCK PRICES

LOGNORMAL MODEL FOR STOCK PRICES MICHAEL J. SHARPE MATHEMATICS DEPARTMENT, UCSD 1. INTRODUCTION What follows is a simple but important model that will be the basis for a later study of stock prices as

### New SAS Procedures for Analysis of Sample Survey Data

New SAS Procedures for Analysis of Sample Survey Data Anthony An and Donna Watts, SAS Institute Inc, Cary, NC Abstract Researchers use sample surveys to obtain information on a wide variety of issues Many

### Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

### 9-3.4 Likelihood ratio test. Neyman-Pearson lemma

9-3.4 Likelihood ratio test Neyman-Pearson lemma 9-1 Hypothesis Testing 9-1.1 Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental

### Sampling Error Estimation in Design-Based Analysis of the PSID Data

Technical Series Paper #11-05 Sampling Error Estimation in Design-Based Analysis of the PSID Data Steven G. Heeringa, Patricia A. Berglund, Azam Khan Survey Research Center, Institute for Social Research

### Annex 5 GUIDELINES FOR SAMPLING AND SURVEYS FOR CDM PROJECT ACTIVITIES AND PROGRAMME OF ACTIVITIES. (Version 02.0) CONTENTS

Page 1 GUIDELINES FOR SAMPLING AND SURVEYS FOR CDM PROJECT ACTIVITIES AND PROGRAMME OF ACTIVITIES (Version 0.0) CONTENTS Paragraphs Page I. Introduction... 1 4 II. Common Types of Sampling Approaches...

### SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

### Sampling Probability and Inference

PART II Sampling Probability and Inference The second part of the book looks into the probabilistic foundation of statistical analysis, which originates in probabilistic sampling, and introduces the reader

### Sampling and Hypothesis Testing

Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

### Gamma Distribution Fitting

Chapter 552 Gamma Distribution Fitting Introduction This module fits the gamma probability distributions to a complete or censored set of individual or grouped data values. It outputs various statistics

### Confidence Intervals and Loss Functions

A.C.E. Revision II PP-42 December 31, 2002 Confidence Intervals and Loss Functions Mary H. Mulry and Randal S. ZuWallack Statistical Research Division and Decennial Statistical Studies Division i CONTENTS

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Survey Data Analysis in Stata

Survey Data Analysis in Stata Jeff Pitblado Associate Director, Statistical Software StataCorp LP Stata Conference DC 2009 J. Pitblado (StataCorp) Survey Data Analysis DC 2009 1 / 44 Outline 1 Types of

### Review of Basic Options Concepts and Terminology

Review of Basic Options Concepts and Terminology March 24, 2005 1 Introduction The purchase of an options contract gives the buyer the right to buy call options contract or sell put options contract some

### ESTIMATION OF THE EFFECTIVE DEGREES OF FREEDOM IN T-TYPE TESTS FOR COMPLEX DATA

m ESTIMATION OF THE EFFECTIVE DEGREES OF FREEDOM IN T-TYPE TESTS FOR COMPLEX DATA Jiahe Qian, Educational Testing Service Rosedale Road, MS 02-T, Princeton, NJ 08541 Key Words" Complex sampling, NAEP data,

### The VAR models discussed so fare are appropriate for modeling I(0) data, like asset returns or growth rates of macroeconomic time series.

Cointegration The VAR models discussed so fare are appropriate for modeling I(0) data, like asset returns or growth rates of macroeconomic time series. Economic theory, however, often implies equilibrium

### ANALYTIC AND REPORTING GUIDELINES

ANALYTIC AND REPORTING GUIDELINES The National Health and Nutrition Examination Survey (NHANES) Last Update: December, 2005 Last Correction, September, 2006 National Center for Health Statistics Centers

### Table 1" Cancer Analysis No Yes Diff. S EE p-value OLS 62.8 61.3 1.5 0.6 0.013. Design- Based 63.6 62.7 0.9 0.9 0.29

Epidemiologic Studies Utilizing Surveys: Accounting for the Sampling Design Edward L. Korn, Barry I. Graubard Edward L. Korn, Biometric Research Branch, National Cancer Institute, EPN-739, Bethesda, MD

### Combining information from different survey samples - a case study with data collected by world wide web and telephone

Combining information from different survey samples - a case study with data collected by world wide web and telephone Magne Aldrin Norwegian Computing Center P.O. Box 114 Blindern N-0314 Oslo Norway E-mail:

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Chapter 19 Statistical analysis of survey data. Abstract

Chapter 9 Statistical analysis of survey data James R. Chromy Research Triangle Institute Research Triangle Park, North Carolina, USA Savitri Abeyasekera The University of Reading Reading, UK Abstract

### A credibility method for profitable cross-selling of insurance products

Submitted to Annals of Actuarial Science manuscript 2 A credibility method for profitable cross-selling of insurance products Fredrik Thuring Faculty of Actuarial Science and Insurance, Cass Business School,

### Modeling and Performance Evaluation of Computer Systems Security Operation 1

Modeling and Performance Evaluation of Computer Systems Security Operation 1 D. Guster 2 St.Cloud State University 3 N.K. Krivulin 4 St.Petersburg State University 5 Abstract A model of computer system

### Non-Parametric Estimation in Survival Models

Non-Parametric Estimation in Survival Models Germán Rodríguez grodri@princeton.edu Spring, 2001; revised Spring 2005 We now discuss the analysis of survival data without parametric assumptions about the

### Selecting Research Participants

C H A P T E R 6 Selecting Research Participants OBJECTIVES After studying this chapter, students should be able to Define the term sampling frame Describe the difference between random sampling and random

### Autocovariance and Autocorrelation

Chapter 3 Autocovariance and Autocorrelation If the {X n } process is weakly stationary, the covariance of X n and X n+k depends only on the lag k. This leads to the following definition of the autocovariance

### SCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING. Self Study Course

SCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING Self Study Course MODULE 17 MATRICES II Module Topics 1. Inverse of matrix using cofactors 2. Sets of linear equations 3. Solution of sets of linear

### Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013

Lecture 11 Shuanglin Shao October 2nd and 7th, 2013 Matrix determinants: addition. Determinants: multiplication. Adjoint of a matrix. Cramer s rule to solve a linear system. Recall that from the previous

### Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

### Chapter 7. One-way ANOVA

Chapter 7 One-way ANOVA One-way ANOVA examines equality of population means for a quantitative outcome and a single categorical explanatory variable with any number of levels. The t-test of Chapter 6 looks

Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

### Lecture Notes Module 1

Lecture Notes Module 1 Study Populations A study population is a clearly defined collection of people, animals, plants, or objects. In psychological research, a study population usually consists of a specific

### Stat 704 Data Analysis I Probability Review

1 / 30 Stat 704 Data Analysis I Probability Review Timothy Hanson Department of Statistics, University of South Carolina Course information 2 / 30 Logistics: Tuesday/Thursday 11:40am to 12:55pm in LeConte

### MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

### Need for Sampling. Very large populations Destructive testing Continuous production process

Chapter 4 Sampling and Estimation Need for Sampling Very large populations Destructive testing Continuous production process The objective of sampling is to draw a valid inference about a population. 4-

### ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

### Monte Carlo Methods in Finance

Author: Yiyang Yang Advisor: Pr. Xiaolin Li, Pr. Zari Rachev Department of Applied Mathematics and Statistics State University of New York at Stony Brook October 2, 2012 Outline Introduction 1 Introduction

### Comparison of Survival Curves

Comparison of Survival Curves We spent the last class looking at some nonparametric approaches for estimating the survival function, Ŝ(t), over time for a single sample of individuals. Now we want to compare

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Confidence Intervals for the Difference Between Two Means

Chapter 47 Confidence Intervals for the Difference Between Two Means Introduction This procedure calculates the sample size necessary to achieve a specified distance from the difference in sample means

### Design-Based Estimators for Snowball Sampling

Design-Based Estimators for Snowball Sampling Termeh Shafie Department of Statistics, Stockholm University SE-106 91 Stockholm, Sweden Abstract Snowball sampling, where existing study subjects recruit

### 1. Let A, B and C are three events such that P(A) = 0.45, P(B) = 0.30, P(C) = 0.35,

1. Let A, B and C are three events such that PA =.4, PB =.3, PC =.3, P A B =.6, P A C =.6, P B C =., P A B C =.7. a Compute P A B, P A C, P B C. b Compute P A B C. c Compute the probability that exactly

### Organizing Your Approach to a Data Analysis

Biost/Stat 578 B: Data Analysis Emerson, September 29, 2003 Handout #1 Organizing Your Approach to a Data Analysis The general theme should be to maximize thinking about the data analysis and to minimize

### LOOKING FOR A GOOD TIME TO BET

LOOKING FOR A GOOD TIME TO BET LAURENT SERLET Abstract. Suppose that the cards of a well shuffled deck of cards are turned up one after another. At any time-but once only- you may bet that the next card

### Systematic Reviews and Meta-analyses

Systematic Reviews and Meta-analyses Introduction A systematic review (also called an overview) attempts to summarize the scientific evidence related to treatment, causation, diagnosis, or prognosis of