Center: Finding the Median. Median. Spread: Home on the Range. Center: Finding the Median (cont.)

Save this PDF as:

WORD PNG TXT JPG

Size: px

Start display at page:

Download "Center: Finding the Median. Median. Spread: Home on the Range. Center: Finding the Median (cont.)"

Willis Tyler
2 years ago
Views:

Transcription

1 Center: Finding the Median When we think of a typical value, we usually look for the center of the distribution. For a unimodal, symmetric distribution, it s easy to find the center it s just the center of symmetry. We could average the minimum and maximum data values (called the midrange) as a measure of center, but the midrange is very sensitive to skewed distributions and outliers. Center: Finding the Median (cont.) A more reasonable choice for center than the midrange is the value with exactly half the data values below it and half above it. This particular value is called the median. The median is the middle data value (once the data values have been ordered) that divides the histogram into two equal areas. The median has the same units as the data. Slide 5-1 Slide 5- Median n + 1 The sample median is the largest observation. n +1 If is not a whole number, the median is the average of the two observations on either side. Spread: Home on the Range When describing a distribution numerically, we always report a measure of its spread along with its center. The range of the data is the difference between the maximum and minimum values: Range = max min. A disadvantage of the range is that a single extreme value can make it very large and, thus, not representative of the data overall. Slide 5-3 Slide 5-4

We could average the minimum and maximum data values (called the midrange) as a measure of center, but the midrange is very sensitive to skewed distributions and outliers.

2 The Interquartile Range The interquartile range (IQR) allows us to ignore extreme data values and concentrate on the middle of the data. To find the IQR, we first need to know what quartiles are Quartiles Quartiles split the data into quarters Lower quartile (Q 1 ) divides bottom half of data into two median of observations below the median Upper quartile (Q 3 ) divides upper half of data into two median of observations above the median The difference between the quartiles is the IQR, so IQR = upper quartile lower quartile. Slide 5-5 Slide 5-6 The Interquartile Range (cont.) The lower and upper quartiles are the 5 th and 75 th percentiles of the data, so The IQR contains the middle 50% of the values of the distribution, as shown in Figure 5.3 from the text: The Five-Number Summary Five number summary { Min, Q 1, Median, Q 3, Max } Example: Slide 5-7 Slide 5-8

median Upper quartile (Q 3 ) divides upper half of data into two median of observations above the median The difference between the quartiles is the IQR, so IQR = upper quartile lower quartile.

3 Boxplots Boxplot A boxplot is a graphical display of the fivenumber summary. The steps involved in constructing a boxplot can also be found on pages of the text. Boxplots are particularly useful when comparing groups. Data 1.5 IQR (pull back until hit observation) Q 1 Med Q IQR (pull back until hit observation) Scale Figure.4.4 Construction of a box plot. From Chance Encounters by C.J. Wild and G.A.F. Seber, John Wiley & Sons, 000. Slide 5-9 Slide 5-10 Construction of Boxplot Comparing Groups With Boxplots Data: breaking strength of wire in kilograms The following set of boxplots compares the effectiveness of various coffee containers: Leaf Unit = 1.0 kg (4) Find Median Find Quartiles Q 1 = Q 3 = Calculate Interquartile range Q 3 -Q 1 = Calculate whisker length 1.5 x (Q 3 -Q 1 ) = What does this graphical display tell you? Slide 5-11 Slide 5-1

4 Construction of a box plot. From Chance Encounters by C.J. Wild and G.A.F. Seber, John Wiley & Sons, 000.

4 Summarizing Symmetric Distributions Medians do a good job of identifying the center of skewed distributions. When we have symmetric data, the mean is a good measure of center. We find the mean by adding up all of the data values and dividing by n, the number of data values we have. Sample Mean average The sample mean is denoted by The sample mean = x Sum of the observations Number of observations Mean (a) (b) (c) Slide 5-13 Figure.4.1 Mechanical construction representing a dot plot: (a) shows a balanced rod while (b) and (c) show unbalanced rods. Slide 5-14 Mean or Median? Regardless of the shape of the distribution, the mean is the point at which a histogram of the data would balance. In symmetric distributions, the mean and median are approximately the same in value, so either measure of center may be used. For skewed data, though, it s better to report the median than the mean as a measure of center. Figure.4. Relationship between mean and median P Med = x (a) Data symmetric about P P Med x (b) Two largest points moved to the right The mean and the median. [Grey disks in (b) are the ``ghosts'' of the points that were moved.] Slide 5-15 From Chance Encounters by C.J. Wild and G.A.F. Seber, John Wiley & Sons, 000. Slide 5-16

Sample Mean average The sample mean is denoted by The sample mean = x Sum of the observations Number of observations Mean (a) (b) (c) Slide 5-13 Figure.4.

5 What About Spread? A more powerful measure of spread than the IQR is the standard deviation, which takes into account how far each data value is from the mean. A deviation is the distance that a data value is from the mean. Since adding all deviations together would total zero, we square each deviation and find an average of sorts for the deviations. Variance The sample variance, denoted by s, is found using the formula s = ( x1 x) + ( x x) ( xn x) 1 = ( x x) i Slide 5-17 Slide 5-18 Sample Standard Deviation Shape, Center, and Spread s x = ( x1 x) + ( x x) ( xn x) 1 = ( x x) In same units as data So preferable to sample variance Equals zero only if all observations identical Sensitive to outliers (extreme observations) Button on calculator learn to use it! Much simpler than applying formula i When telling about a quantitative variable, always report the shape of its distribution, along with a center and a spread. If the shape is skewed, report the median and IQR. If the shape is symmetric, report the mean and standard deviation and possibly the median and IQR as well. Slide 5-19 Slide 5-0

Variance The sample variance, denoted by s, is found using the formula s = ( x1 x) + ( x x) +.

6 What About Outliers? If there are any clear outliers and you are reporting the mean and standard deviation, report them with the outliers present and with the outliers removed. The differences may be quite revealing. Note: The median and IQR are not likely to be affected by the outliers. What Can Go Wrong? Do a reality check don t let technology do your thinking for you. Don t forget to sort the values before finding the median or percentiles. Don t compute numerical summaries of a categorical variable. Watch out for multiple modes multiple modes might indicate multiple groups in your data. Slide 5-1 Slide 5- What Can Go Wrong? (cont.) Be aware of slightly different methods different statistics packages and calculators may give you different answers for the same data. Beware of outliers. Make a picture (make a picture, make a picture). Be careful when comparing groups that have very different spreads. So What Do We Know? We describe distributions in terms of shape, center, and spread. For symmetric distributions, it s safe to use the mean and standard deviation; for skewed distributions, it s better to use the median and interquartile range. Always make a picture don t make judgments about which measures of center and spread to use by just looking at the data. Slide 5-3 Slide 5-4

Don t forget to sort the values before finding the median or percentiles. Don t compute numerical summaries of a categorical variable.

Data Mining Part 2. Data Understanding and Preparation 2.1 Data Understanding Spring 2010

Data Mining Part 2. Data Understanding and Preparation 2.1 Data Understanding Spring 2010 Data Mining Part 2. and Preparation 2.1 Spring 2010 Instructor: Dr. Masoud Yaghini Introduction Outline Introduction Measuring the Central Tendency Measuring the Dispersion of Data Graphic Displays References