Skewness of Data, T-Test, and Analysis of Variance Scribed By: Vincent Ciaramella In this lecture we will go into some detail about a few forms of data analysis. Experiment Explanation Last lecture, we looked at some example data for a hypothetical experiment in which several students drew a circle using either a mouse or a leap motion (gestural) interface. The time taken to complete this task was recorded. The data was recorded into a table whose columns were named as follows: Participant, Interface, Time Taken, and Contact Interface. Being that the software used to analyze this data will not appropriately handle 'leap' vs. 'mouse' we added the 'Contact Interface' column to represent this via 1s and 0s. To clarify the data addition, I will supply an additional example. For an experiment that I am currently working on, we have datasets for different musical instruments. They were previously encoded as piano, guitar, drum etc. and then later correlated to an additional column which would hold piano as the number 0, guitar as 1, etc. This extra (numerical) column will make working with R an easier task. Analysis Explanation The focus of our analysis will be to understand whether the independent variable (type of interface) has an effect on the dependent variable (time taken). Thus, we would like to see if using the mouse led to a slower time taken than when using the leap motion device. There are a few different ways in which we could test for this. The two ways we will cover are T-tests and one way over analysis of variance (ANOVA) Think of these as tools. You should reference a good statistics book to become more familiar, but here we will just cover how to analyze data and use the results. Analysis using R The work below was done in R. The R statements will be prefixed with '> ' while the comments and explanations will be prefixed by '// '. // Clear the screen and any previous data > rm(list=ls())
> ls() // Next we load the data. > file.name<-file.choose() // The previous command will bring about a dialogue used to select the data. // Once a selection is made, the data will be read in. > my.data<-read.csv(file.name,row.names=null) // Now we will view the data using the following command. > my.data // Next we will run a t-test. Last class we found that our means are different. Now we want to see if that difference arose by chance or is a systematic different (meaning that it is a result of the fact that there are two different types of interfaces). The t-test allows us to test the likelihood that a difference of means is due to the dependent variable (or a fluke). // We run a t-test on the third column (time taken) grouped by the second column (interface type) as shown below. > t.test(my.data[,3]~my.data[,2]) // If instead we want to run a paired t-test then we would run the following command. > t.test(my.data[,3]~my.data[,2],paired=true) T-Test Now we will take a moment to look at the t-test results. The p value returned here tells us the probability that the difference in means (in our current data set) is by chance. Basically the p value is the probability that our null hypothesis (that there is no statistically significant difference between the means) is true. For the test data, the p value is 0.001163. This low p value (< 0.05) tells us that we will likely see this change occurring in subsequent experiments. Thus we are confident that there is a
difference in the time taken to do a certain task between these two types of interfaces. The way we want to report this finding would be to say: "There was a significant difference between the type taken on the leap motion controller compared to the mouse. (t(3) = -12.2816, df=3, p- value=0.001163). The mean value for leap = ##, the mean value for mouse = ##." Basically what we were saying is that greater than 95 percent of the time we expect to see a difference of means. Degrees of Freedom Now a note on the t(3) and df=3 from the results above. This is how we express the degrees of freedom, which is a parameter in the t test. From a practitioner s point of view, it is related to the number of participants you had. The reason why you would want to report this is that if you have very large degree of freedom then you would see a statistical difference in the result. So what people want to see is that the degrees of freedom is at an acceptable level for the community. Acceptable p values It is worth noting that these p values may vary across different fields. For example, in medical research the p values are typically higher. In social behavioral research p values that are less than 0.05 are acceptable. Generally the rule of thumb in terms of an acceptable level is dictated by previous research. As an additional example say you are conducting an experiment to build upon previous work. If the previous work utilized a t-test with a specified p value then your work should use either the same or a more strict (smaller) required p value to reject the null. A note on t-tests: There are many flavors of t-tests which you should consult a statistics book in order to identify the most relevant to your experiment. Examples would be paired t-tests (for within subjects), two tailed t-tests, one tailed t-tests, etc. Analysis of Variance (ANOVA) Another method that is equivalent to the t-test is known as an analysis of variance (ANOVA). We will go back to our sample data in R to cover an example of such an analysis.
// Here we run an analysis of variances on the third column ordered by the second column > result = aov(my.data[,3]~my.data[,2]) // This will give us a table as results. Once again we look at the p value (under Pr(>F)) to test a null hypothesis. A p value above a certain threshold will indicate a significant difference. The degrees of freedom for this test are n- 1. With the resulting p value of 0.00637 we can conclude that there is a significant difference between the two means. The way we would report this is: "There is a statistically significant difference in the time taken by a mouse relative to LEAP interface. (F(subscript 1,3)= 16.8, p < 0.05 ). The mean value for leap = ##, the mean value for mouse = ##." If the p value had been greater than 0.05 (or our given threshold) then we could not conclude that a statistically significant difference was shown (between the means). Publishing Results Typically it would be hard to publish a paper that doesn t elicit significant results. What typically happens is you ran a series of results and you explain why you think that happened. The paper is basically saying here is what we did in our lab and here is what we learned from it. By and large it is much easier to publish with significant results. Usually you would iterate your design a few times if you did not get significant results. Higher Independent Variable Levels Now we will cover the case where there are more levels to our independent variable by augmenting our previous example. Once again we go back to R. // Here we cleared the previous data and opened a new set. The new set contains data for a mouse, touchpad, and leap interface > rm(list=ls()) > file.name<- file.choose() > my.data<-read.csv(file.name,row.names=null)
> my.data // This will show the same format as before but with mouse, touchpad, and leap in the interface column. // Now we run the analysis of variances with those three types of gui data present. > result=aov(my.data[,3]~my.data[,2]) > summary(result) Once again our p value is less than 0.05. These results tell us that there is at least one difference between two of the three means where at least one of them is statistically significant. We still don't know which one it is but we know it exists. To report this finding, we say: "There is a significant main effect of interface on time taken.(f(subscript2,22)=16.8, p<0.00738). The mean value for leap = ##, the mean value for mouse = #, the mean value for touchpad is ##". Further analysis will need to be conducted in order to find which of these differences is significant. For this task we will use the Tukey HSD (Honest Significant Differences) function. // Now we use Tukey HSD (Honest Significant Differences). This is what we use when there are more than two conditions. > Tukey HSD(result,ordered=TRUE) // These results will contain a p value for each difference. Analysis with Multiple Independent Variables If the current experiment was augmented in that there were two specific tasks (each of which using all three types of interfaces). These tasks could be writing and drawing. We may want to analyze the effect of two factors at the same time. To do this we would use a 2-way ANOVA analysis. As an additional example consider an experiment where different footwear are being tested. One dependent variable would be the foot ware type (low top versus high top shoe) while another dependent variable (or factor) would be the task (a jumping versus a running task). In order to analyze the effect of the two factors at the same time, we would want to use a two way ANOVA analysis.