Descriptive Statistics III. Transformations

Transcription

1 MAT 2379 (Spring 2012) Descriptive Statistics III Transformations Often in practice, we transform our data by applying a transformation (i.e. using a function) on the numerical variable. In these notes we investigate the effects of these transformation on the distribution. We will consider two types of transformations : A linear transformation : A logarithmic transformation : y i = a x i + b, for i = 1,..., n. y i = ln(x i ), for i = 1,..., n. Linear Transformation We will study the effects of a linear transformation : y i = a x i + b, for i = 1,..., n. The mean is linear, that is the mean of the transformed variable is the linear transformation of the mean of the original variable : y = a x + b. Proof : y = yi n = (a xi + b) n = a ( x i ) + n b n = a x + b. 1

2 The variance is not linear. Here is the variance of the transformed variable : s 2 y = a 2 s 2 x. Proof : s 2 (yi y) 2 y = [a xi + b (ax + b)] 2 = [a xi (ax)] 2 = a 2 (x i x) 2 = = a 2 (xi x) 2 = a 2 s 2 x Consequences of the linear transformation y i = a x i + b : 1. mean : y = a x + b 2. variance : s 2 y = a 2 s 2 x 3. standard deviation : s y = a 2 s 2 x = a s 2 x 4. If a > 1, then y is more variable than x. If 0 < a < 1, then y is less variable than x. Example 6 : Consider the following 7 temperatures in Fahrenheit. They represent the average daily temperature on San Cristóbal island, for 7 days during the dry season. 73.4, 68.9, 73.4, 66.2, 69.8, 71.6, Let x be the average daily temperature in Fahrenheit. Using Minitab, we obtained the following summary statistics. 2

3 Let y be the average daily temperate in Celsius. Then, y = (5/9) (x 32). Note that this is a linear transformation. Compute the mean and the standard deviation of temperature in Celsius. 3

4 Transforming a variable in Minitab Here is the command to transform the values in C1 from Example 1 into Celcius. The result is stored in column C2. let c2=(c1-32)*5/9. Then, we use the command DESCRIBE to obtain descriptive statistics for both C1 and C2. Here is the result. 4

5 Logarithmic Transformation Often in nature, there are distributions that are strongly skewed to the right. The mean and the standard deviation are often not a good measure of center and of dispersion, respectively, in these cases. A remedial measure is to apply a transformation to the variable to make distribution approximately symmetric. A transformation that works often (but not always) is a logarithmic transformation. y i = ln(x i ), for i = 1,..., n. Example 7 : Consider n = 250 survival times after identifying a certain type of cancer. The data is found in SURVIVALTIMES.txt. We will assume that these data are in column C1 and we will produce the log-times and store them in column C2. Here is the command. MTB > Let c2 = ln(c1) Note : We have given names to these variables in the worksheet : Survival Times (in months) et Survival Times (in log(months)). We will produce a histogram for each variable. Here is the command. MTB > histogram c1 c2. Here are the histograms. 5

6 Remarks : The distribution of the survival times is highly skewed to the right. However, the distribution of the logarithmic times is approximately symmetric. Example 8 : Consider the survival times (in months) of Example 7. We have the survival times (in months) in column C1 and the logarithmic times in column C2. We will compute the mean and the standard deviation for both of these variables. Here are the commands. MTB > describe c1 c2; SUBC> mean; SUBC> stdeviation; SUBC> count. Here is the output. 6

7 Since the distribution of the log-times is approximately symmetric, then the mean and the standard deviation of the log-times are meaningful measures of the central tendency and the dispersion of these values. But in general, we are interested in the survival times in months and not the logarithmic times. We can apply the inverse function of the logarithm, that is the exponential function to obtain measures on the original scale. But we must be careful in the interpretation. Compute the exponential of the mean and of the standard deviation of the logarithmic times. We get e y = e = months and e sy = e = months. Remark : e y x and e sy s x. Geometric mean and standard deviation When you apply a logarithmic transformation to the variable x, that you compute the mean of the logarithmic values and that you apply the exponential transformation to the latter mean, we get the geometric mean of the variable x : g = e y = e (1/n) ln(x i ) = e ln(x i ) 1/n = (x 1 x n ) 1/n. The corresponding measure of dispersion, i.e. e sy, is the geometric standard deviation of the variable x, that is defined as the exponential of the standard deviation of the logarithmic values. 7

8 Properties of the logarithmic transformation y i = ln(x i ) : The logarithmic transformation can sometimes transform a distribution that is highly skewed to the right, to an approximately symmetric distribution. If the distribution of the logarithmic values is approximately symmetric, then we can use the mean of the logs and the standard deviation of logs values to measure the center and the dispersion of distribution of logarithmic values. The exponential of the mean of the logarithmic values is the geometric mean of the original variable : g = e y. The exponential of the standard deviation of the original values is the geometric standard deviation of the original variable, that we denote e sy. 8