Graphs. Exploratory data analysis. Graphs. Standard forms. A graph is a suitable way of representing data if:

Transcription

1 Graphs Exploratory data analysis Dr. David Lucy Lancaster University A graph is a suitable way of representing data if: A line or area can represent the quantities in the data in some way. Several standard forms can be used. Standard forms are not the only forms. You can make up your own if you please R very good for this. Exploratory data analysis p.1/36 Exploratory data analysis p.3/36 Graphs Standard forms Graphics are a very important part of making sense of data: Allow the researcher to compare quantities easily and simply by comparing lengths and/or areas. Many humans adapted to view quantities rather than number. Immediate impact. Suggests ideas for further work. For many scientists this is their main form of anlysis - some of the worlds best science has been done purely by graphs. There are three standard types of graph: 1. histogram - used to examine the distribution of a set of observations - can be used to compare distributions between sets of observations - observations may be discrete (underlying continuous) and continuous, 2. scatterplot - use to look for relationships between different continuous variables, 3. boxplot - sometimes called box and whiskers plot - use to compare distributions of continuous variables which is equivalent to looking for relationships between factors and continuous variables. Exploratory data analysis p.2/36 Exploratory data analysis p.4/36

2 Histograms Histograms Partial Full Partial Full Exploratory data analysis p.5/36 Exploratory data analysis p.7/36 Histograms Exercise Do not confuse histigrams with barcharts: Histograms have the area proportional to the quantity of interest: not necessarity equal column widths, although most are. Rescale the full histogram so the bars sum to one? Number of runs in 109 observations Barcharts have the column height proportional to the quantity of interest. Exploratory data analysis p.6/36 Exploratory data analysis p.8/36

3 Exercise Histogram comparison = divide each frequency by 109: Ladybower Reservoir Number of runs in 109 observations Normalised Process sometimes called normalisation by scientists Daily max ozone Daily max ozone Exploratory data analysis p.9/36 Exploratory data analysis p.11/36 Scaled histograms Histogram comparison Differences Partial Full Daily max ozone Exploratory data analysis p.10/36 Exploratory data analysis p.12/36

4 Histogram problems Kernal density estimates probability density Summer daily maxima Summer daily maxima Exploratory data analysis p.13/ x Exploratory data analysis p.15/36 Kernal density estimates Cumulative distributions Ladybower Recall the cumulative distribution function (c.d.f.) of a random variable X: F(x) = P(X x) How can we estimate this from a finite number of observations? Ozone (ppb) Exploratory data analysis p.14/36 Exploratory data analysis p.16/36

5 Cumulative distributions Cumulative distributions Let us assume That our variables X 1,...,X n are independent and identically distributed (i.i.d.) They are replicates of a random variable X which has cumulative distribution function F. We can denote by x 1,...,x n, the observed values of X 1,...,X n. The empirical c.d.f is a proper distribution function and has the following properties: F(x) is a step function with jumps at the data points; F(x) = 1 if x max(x 1,...,x n ); F(x) = 0 if x < min(x 1,...,x n ). Exploratory data analysis p.17/36 Exploratory data analysis p.19/36 Cumulative distributions Cumulative distributions The empirical cumulative distribution function (c.d.f.) is defined as: F(x) = 1 n n (num of x i=1 i x) = Π(x i x) n where : Π(x i x) = { 1 if x i x 0 if x i > 0 To construct: Take the observed values and order them so that the smallest one comes first. Label these ordered values x (1),x (2),,x (n) so that x (1) x (2) x (n). Then the kth ordered point x (k) is the k/n th quantile. Exploratory data analysis p.18/36 Exploratory data analysis p.20/36

6 Exercise Exercise For the observations {1, 2, 2, 3, 4}, find F(x) and sketch the plot. x F(x) density x Exploratory data analysis p.21/36 Exploratory data analysis p.23/36 Exercise Exercise For the observations {1, 2, 2, 3, 4}, find F(x) and sketch the plot. x n(x i ) x i F(x) 0 0 1/5 1/5 3/5 3/5 4/5 4/5 5/5 5/5 5/5 F(x) Construct the cdf for the first 20 points from the Summer ozone measurements and sketch it These are: At each sorted data point we have a jump of i/n, which is 1/20 as n = 20. Exploratory data analysis p.22/36 Exploratory data analysis p.24/36

7 Exercise Scatterplots Fn(x) Scatterplots look at the relationship between continuous variables. Usually they project two dimensions onto two dimensions. Several ways of representing three dimensions. Scatterplots are the mainstay of physical sciences Summer daily maxima Exploratory data analysis p.25/36 Exploratory data analysis p.27/36 Summer ozone Scatterplots Summer Winter Fn(x) Summer daily maxima NO NO2 Exploratory data analysis p.26/36 Exploratory data analysis p.28/36

8 Scatterplots Independence peripheral COHb saturation level Conditional probabilities were introduced in Math104: If A and B are two events then, as long as P(B) > 0, the conditional probability of A given B is written as P(A B) and calculated from: P(A B) = P(A B). P(B) heart COHb saturation level Exploratory data analysis p.29/36 Exploratory data analysis p.31/36 Independence Independence Scatterplots can be used to look for dependence between continuous variables. They can also be useful to identify situations in which variables appear to be independent. If two variables are independent, then the distribution of one variable will look the same regardless of the value of the other variable. This is what the ozone versus NO 2 above plots looked like. We can look for some structure in our data: including the dependence of one variable on another, by examining conditional distributions of some subsets of our data. Do this by seperating the data by some defined criterion, and plotting the subsets. Exploratory data analysis p.30/36 Exploratory data analysis p.32/36

9 Exercise Boxplots Summer Ozone NO2 <= 40 Winter Ozone NO2 <=40 Summer Winter Summer Ozone 40<NO2<= Winter Ozone 40<NO2<= Ladybower Ladybower. Exploratory data analysis p.33/36 Exploratory data analysis p.35/36 Boxplots Next session The third of the standard forms for graphs: Similar to multiple histograms. Examine distribution of continuous variable. For different levels of a discrete variable. The discrete variable can be ordered, or nominal. Next time we shall: 1. take a look at boxplots, 2. learn about some of the classic plots from history, 3. find out what makes a good graph, 4. look at some non-standard forms of graphs. Exploratory data analysis p.34/36 Exploratory data analysis p.36/36