Methods for Describing Sets of Data

Transcription

1 Chapter 2 Methods for Describing Sets of Data 2.1 Describing Qualitative Data Definition 2.1 Class: A class is one of the categories into which qualitative data can be classified. Definition 2.2 Class Frequency: The class frequency is the number of observations in the data set falling in a particular class. Definition 2.3 Class Relative Frequency: The class relative frequency (RF) is the class frequency divided by the total number of observations in the data set. That is class relative frequency = class frequency n Definition 2.4 Class Percentage: The class percentage is the relative frequency multiplied by 100. That is Class precentage= class relative frequency 100 Example 2.1, page Graphical Methods for Describing Quantitative Data Data can be described by graphically and numerically. This section will discuss about the graphical representation of the data. For describing, summarizing and detecting patterns of a data set, one can use the following three graphical methods. 1. Dot plots: The dot plot condenses the data by grouping all values that are same together in the plot. 2. Stem-and-leaf displays (stem plots): The stem-and-leaf plot display condenses the data by grouping all data with the same stem together in the graph. 6

2 3. Histograms: The histogram condenses the data by grouping similar data values in the same class in the graph. Since most of the statistical software packages can be used to construct these plots, we will focus on their interpretations rather than their constructions. Suppose a financial analyst is interested in the amount of resources spent by computer hardware and software companies on research and development (R&D). She samples 50 of these high-technology firms and calculates the amount each spent last year on R&D as percentage of their total revenue. The results are given (see Table 2.2, page 42) are as follows: The ordered data are as follows: Dot plots (Figure 2.8, page 42) 2. Stem-and-leaf displays (stem plots), (Figure 2.9, page 43). Splus software produces the following stem-plot. N = 50 Median = 8.05, Quartiles = 7.1, 9.6 Decimal point is at the colon 5 : : : : : : : : 13 : 255 7

3 3. Histograms (Figure 2.10, page 44). Using Splus, we have the following histogram for R&D Data Discussion about Dot plot, Stem plot and Histogram: Histogram provides a good visual descriptions of data sets, but it can not identify individual measurements. In contrast, each of the original measurements is visible to some extend in dot and stem plot. The stem plot arranges the data in ascending order, so, it is easy to locate the individuals (measurements). However, it can become cumbersome or diminishing the usefulness of the visual display for a very large data sets. Histogram is very useful for describing the large data sets when the overall shape of the distribution of measurements is more important than the identification of individual measurements. For large data set use histogram and small data set use stem-and leaf or dot plot. Among the three methods, histogram is the most useful display. Principles for constructing a frequency distribution 1. Determine the number of classes (see page 45). Approximate number of classes, k =1+3.3 log(n). In general, 5 k Determine the class width Largest observation - Smallest observation Class width = Number of classes Classes of equal width allow us to make uniform comparisons of the class frequencies. 3. Locate the class boundaries: The class boundaries are chosen so that each measurements can fall one and only one subinterval. When the classes are formed, the measurements are categorized according to the class into which they fall, and a graph resembling a bar graph is drawn. This graph is called a relative frequency histogram. RAD 8

4 The class intervals, frequencies, and relative frequency (RF) for the 50 R&D data are given in the following table (Table 2.3, page 44) Table 2.3: Frequency Distribution Table Class Class Interval Class Frequency Class RF /50= /50= /50= /50= /50= /50= /50= /50= /50=0.06 Total Relative frequency histogram: The relative frequency histogram for a quantitative data set is a bar graph in which the height of the bar represents the proportion or relative frequency of occurrence for a particular class or subintervals of variable beings measured. The classes or subintervals are plotted along the horizontal axis. Frequency distribution: The relative frequency histogram is often called a frequency distribution, because, it shows the manner in which the data are distributed along the horizontal axis of the graph. See the frequency distribution of R&D data in Table 2.3, page 44. By summing the relative frequencies in the intervals from to , we can see that 82% of the R&D measurements are between 6.0 to Similarly, 6% of the R&D measurements are over Comments about histogram: The most significant features of the sample frequency histogram is that it provides information about the population frequency histogram that describes the population. Note that different samples from the same population will result in different sample histograms, even when the class boundaries remain fixed. However, we would expect the sample and population frequency histograms to be similar. The degree of resemblance will increase as more and more data are added to the sample, that is when sample is large. Note: A histogram would be symmetric or asymmetric (skewed to right or skewed to left). Different shape of the histograms are provided in Figure 2.1. Exercise 2.16, page 48. Exercise 2.17, page 48. Exercise 2.20, page 48. 9

5 Data Data Data Data 2.3 Summation Notation Figure 2.1: Different shapes of histograms Suppose we have a set of n measurements as follows x 1,x 2,x 3,...x n Then n x i = x 1 + x 2 + x x n (sum of x s) n x 2 i = x2 1 + x2 2 + x x2 n (sum of x 2 ) Suppose we have the following 5 measurements 5, 3, 8, 5, 4 Then 5 x i = x 1 + x 2 + x 3 + x 4 + x 5 = =25 5 x 2 i = x x x x x 2 5 = 139 Exercise 2.33, page 54 10

6 2.4 Numerical Measures of Central Tendency This and the following section will discuss about the numerical measures of the data. The central tendency of the data gives the center of the data. Definition 2.4 Sample Mean: The sample mean of a set of n measurements (observations) is equal to the sum of the measurements divided by n. The mean of a set of n measurements x 1,x 2,...,x n is denoted by x and defined as x = x 1 + x x n n = n x i. n Population Mean: The population mean of a set of N measurements (observations) is equal to the sum of the measurements divided by N. The mean of a set of N measurements x 1,x 2,...,x N is denoted by μ and defined as μ = x 1 + x x N N = N x i N. Example 2.3, page 55: Find the mean of the following 5 measurements: 5, 3, 8, 5, 6. Example 2.4, page 55. Calculate the sample mean for the R & D expenditure percentages of the 50 companies given in Table 2.2. Definition 2.5 Sample Median: The median is a measure of central tendency that divides the data into two equal parts, half below the median and half above. If the number of measurements is even, the median is halfway between the two central values. The sample median is denoted by M. Example 2.5, page 57: Consider the following sample of n = 7 measurements: 5, 7, 4, 5, 20, 6, 2 a. Calculate the median M. b. Eliminate the last measurement (2) and calculate the median for n = 6 measurements Mean or Median? The mean is very sensitive to one or more outliers (extreme values), while the median is not. Generally, the median will provide a better description of the center of a data set if the distribution of the data is highly skewed (to left or right). However, mean will provide a better description of the center of a data set if the distribution of the data is symmetric. Extra Example 1: Consider the sample measurements of the following data sets: 2, 3, 9, 7, 11, 17. First, find the mean and median of the above data set. Then replace 17 with 117 and again calculate the mean and median. You will find your answer for the above comment mean or median?. Definition 2.6 Skewness: A data set is said to be skewed if one tail of the distribution has more extreme observations than the other tail. There are two kind of skewness: right 11

7 skewed (right tail of the distribution has more extreme observations than the left tail) and left skewed (left tail of the distribution has more extreme observations than the right tail). Relationship between Mean (μ) and Median (M). See Figures on page For a right skewed histogram: Mean Median. 2. For a symmetric histogram: Mean=Median. 3. For a left skewed histogram: Mean Median. Definition 2.7 Mode: The mode is the measurement that occurs most frequently in the data set. Example 2.7, page 58: Consider the sample measurements: 8, 7, 9, 6, 8, 10, 9, 9, 5, 7 and find the mode of the data set. Exercise 2.41, page 61: Calculate the mode, mean and median of the following data: Exercise 2.46, page , 10, 15, 13, 17, 15, 12, 15, 18, 16, Numerical Measures of Variability (Dispersion) Measures of central tendency (mean, median and mode) provide only a partial description of a quantitative data set. The description is incomplete without a measure of the variability, or dispersion or spread of the data set. Both central and variablity of the data set can help us to visualize the shape of a data set as well as its extreme values. Definition 2.8 Range: The range is the simplest measure of variability to calculate. The range is simply the highest score minus the lowest score. Range= Largest measurement (Maximum) - Smallest measurement (Minimum) Population variance: The population variance of N measurements x 1,x 2,...,x N is defined to be the average of the squares of the deviations of the measurements about their mean μ. The population variance is denoted by σ 2 and defined as σ 2 = 1 N N (x i μ) 2 Definition 2.9 Sample variance: The sample variance of n measurements x 1,x 2,...,x n is defined to be the average of the squares of the deviations of the measurements about the sample mean x. The sample variance is denoted by s 2 and defined as s 2 = 1 n 1 n (x i x) 2 12

8 The computing formula or short-cut method for s 2 is s 2 = 1 ( n ) x 2 i n 1 n x2 or s 2 = 1 ( n x 2 i ( n x i ) 2 ) n 1 n Definition 2.10 Standard deviation: The square root of the variance is called the standard deviation (SD). For example the sample standard deviation is s = s 2 = 1 n 1 n (x i x) 2 Note: Unlike the variance, the SD is expressed in the original units of measurements. For example, if the original measurements are in lb, the variance is expressed in units lb squared but the SD as well as mean are expressed in lb. Example 2.9, page 67: Calculate the variance and SD of the measurements 2, 3, 3, 3, and 4. Exercise 2.64, page Interpreting the Standard Deviation Chebyshev s Rule: Chebyshev s rule can be applied to any set of measurements. However, this theorem is very useful to describe the asymmetric distributions (right skewed or left skewed). Given a number k greater than or equal to 1 and a set of n measurements x 1,x 2,...,x n, at least ( ) 1 1 k of the measurements lie within k standard deviations of 2 their ( mean. ) That is 1 1 k 100% of the measurements will lie between μ kσ to μ + kσ. 2 More specifically, for any distribution at least 0% (or very few) of the measurements will fall between x s to x + s. at least 75% of the measurements will fall between x 2s to x +2s. at least 89% of the measurements will fall between x 3s to x +3s. Empirical Rule: Given a distribution of measurements that is approximately bell -shaped (or mound shaped), then approximately 68% of the measurements will fall between x s to x + s. approximately 95% of the measurements will fall between x 2s to x +2s. approximately 99.7% of the measurements will fall between x 3s to x +3s. 13

9 What is the fundamental difference between Chebyshev s rule and empirical Rule? Example 2.11, page 71. Exercise 2.154, page 107. Approximate standard deviation: s Range 4 (for mound shaped data) Approximate standard deviation: s Range 6 (for skewed data) Example 2.12, page 72. Example 2.13, page 73. Exercise 2.76 page Numerical Measures of Relative Standing When we want to know the position of an observation relative to others in a set of data, then the measures of relative standing is useful. Definition 211: Let x 1,x 2,...,x n be a set of n measurements arranged in increasing order. The pth percentile is the value of x such that at most p percent of the measurements are less than that values of x and at most (1 p) percent greater. Example 2.14, page 77. Definition 212 z-score: A z-score represents the distance between an observation and the mean of the data set expressed in standard deviation. The sample z-score corresponding to an observation x is a measure of relative standing is defined by z score = x x s The population z-score for a measurement x is z score = x μ σ Example 2.15, page 78. Interpretation of z-scores for the Mound-Shaped Distributions of Data 1. Approximately 68% of the measurements will have a z-score between -1 to Approximately 95% of the measurements will have a z-score between -2 to Approximately 99.7% of the measurements will have a z-score between -3 to +3 Exercise 2.89, page

10 2.8 Not in the Syllabus 2.9 Not in the Syllabus 2.10 Not in the Syllabus 2.11 Distorting the Truth with Descriptive Techniques A picture may be worth a thousand words. Graphical Distortions: See Figure 2.38, page 97. Misleading numerical descriptive statistics: Example 2.21, page