IS463 Introduction to Data Mining Semester 1, Academic year Tutorial # 2

Transcription

1 IS463 Introduction to Data Mining Semester 1, Academic year Tutorial # 2 Activity 1: Classify the following attributes as qualitative (nominal or ordinal) or quantitative (interval or ratio), also classify them as binary, discrete, and continuous: Age in years. Outlook for weather data (sunny, overcast, rainy) Temperature in weather data. (Hot, mild, cool) Angles as measured in degrees between 0 and 360. Bronze, Silver, and Gold medals as awarded in the Olympics. Height above sea level. Number of patients in a hospital. Calendar dates. Ability to pass light in terms of the following values: opaque, translucent, and transparent. Distance from the center of campus. Density of a substance in grams per cubic centimeter. Coat check number. Street Numbers. The answers are between the brackets: Age in years.[discrete, quantitative, ratio] Outlook for weather data (sunny, overcast, rainy). [Discrete, qualitative, nominal] Temperature in weather data. (Hot, mild, cool)[discrete, qualitative, ordinal] Angles as measured in degrees between 0 and 360.[Continuous, quantitative, interval] Bronze, Silver, and Gold medals as awarded in the Olympics.[Discrete, qualitative, ordinal] Height above sea level.[continuous, quantitative, interval] Number of patients in a hospital.[discrete, quantitative, ratio] Calendar dates.[discrete, quantitative, interval] Ability to pass light in terms of the following values: opaque, translucent, and transparent.[discrete, qualitative, ordinal]

2 Distance from the center of campus.[continuous, quantitative, ratio] Density of a substance in grams per cubic centimeter.[continuous, quantitative, ratio] Coat check number.[discrete, qualitative, nominal] Street Numbers.[Discrete, qualitative, nominal; unless they can be used for ordering then it would be ordinal]. Activity 2: From those two given tables, give example for each type of attribute with a brief description. Show how may the same attribute have different types based on how it is being represented. Table 1: Outlook Temperature Humidity Windy Play Sunny Hot High False No Sunny Hot High True No Overcast Hot High False Yes Rainy Mild Normal True Yes Table 2: Outlook Temperature (F) Humidity Windy Play Sunny False No Sunny True No Overcast False Yes Rainy True Yes Nominal: Outlook and windy columns in both tables. Ordinal: Temperature and humidity in table 1. Interval: Temperature and humidity in table 2. Ratio: None. Activity 3: Give three examples of data quality problems with a brief description by examples. Then, Mention some methods for handling the missing data.

3 Noise and outliers: Noise refers to modification of original values Examples: distortion of a person s voice when talkingon a poor phone and snow on television screen. Outliers are data objects with characteristics that are considerably different than most of the other data objects in the data set. Missing values: - Information is not collected(e.g., people decline to give their age and weight). - Attributes may not be applicable to all cases (e.g., annual income is not applicable to children). Duplicate data Data set may include data objects that are duplicates, or almost duplicates of one another. Example: Same person with multiple addresses. Some methods for handling the missing data Eliminate Data Objects Estimate Missing Values Ignore the Missing Value During Analysis Replace with all possible values (weighted by their probabilities) Activity 4: Given this scale for a candy bar {Poor, fair, OK, good, wonderful}. Find d (good, wonderful), d (fair, wonderful), and s as well. The type of the attributes is ordinal. Thus, the values of the ordinal attribute are mapped to successive integers, {poor = 0, fair = 1, OK = 2, good = 3, wonderful = 4}, then: d (good, wonderful) = (4 3) / 4 = 0.25 s = 1 d = = 0.75 d (fair, wonderful) = (4 1 ) / 4 = 0.5 s = = 0.5

4 Activity 5: Given those two points: p (0, 2), q (5, 1). Calculate the Euclidean Distance between them. What about d (q, p)? dist = d (p, q) = d (q, p) = d (p, q) = Activity 6: Considering those two binary vectors: X = (1, 1, 0, 0, 0, 0, 0, 0, 0, 0) Y = (0, 1, 0, 0, 0, 0, 1, 0, 0, 1) Calculate SMC and J. f00 = the number of attribute where x is 0 and y is 0 f01 = the number of attribute where x is 0 and y is 1 f10 = the number of attribute where x is 1 and y is 0 f11 = the number of attribute where x is 1 and y is 1 In this example, the value of those quantities are: f00 = 6,f01 = 2, f10 = 1, f11 = 1 SMC = number of matching attribute values / number of attributes = ( f11+ f00) / ( f00 + f01 + f10 + f11 ) = ( ) / ( ) = 0.7 J = number of matching presence / number of attributes not involved on 00 matches

5 = f11 / ( f01+ f10 + f11 ) = 1 / ( ) = 0.25 Activity 7: -Considering those two document vectors: X = (3, 2, 0, 5, 0, 0, 0, 2, 0, 0) Y = (1, 0, 0, 0, 0, 0, 0, 1, 0, 2) Calculate the cosine similarity. cos = x y / x y, where indicates the vector dot product, n k=1 x y = xkyk, n2 k=1 x is the length of vector x, x = xk = x x x y = 3 * * * * * * * * * * 2 = 5 x = (3 * * * * * * * * * * 0) = 6.48 y = (1 * * * * * * * * * * 2) = 2.24 cos(x, y) = 0.31 Activity 8: Correlation For the following two vectors, x and y, find the correlation. X = (4, 4, 3, 1, 0, 5, 3, 4, 4, 3) Y = (3, 4, 3, 3, 2, 4, 4, 4, 5, 4)

6 r = correlation (x, y) n = number of values S. D. (x) = standard deviation (x) S. D. (y) = standard deviation (y) n x y xy x 2 y Sum corr (x, y) = ( 10 * 120 ) ( 31 * 36 ) / [( 10 * )* ( 10 * )] = 0.73