Quantitative Analysis of Reading Comprehension and Reading Speed Based on Serif and Sans-Serif Fonts Abstract Taylor Burke Clemson University tdburke@clemson.edu Our work investigates the effect of using the two general categories of fonts, serif fonts and sans-serif fonts, on the rate at which a person reads and on how much of the material a person comprehends. We have conducted an experiment in which each participant is given two articles, each using a different font, and asked to answer a short questionnaire between each article. After analyzing the data, we have come to the conclusion that serif vs sans-serif fonts produce no notable effects on reading speed or comprehension. Keywords: reading; reading speed; serif; sans-serif; font; comprehension 1 Introduction When people read, whether it be by computer, newspaper, or book, words are usually displayed in one of two broad categories of fonts serif fonts, and sans-serif fonts. Traditionally, serifs were said to be used in an attempt to help the horizontal flow of reading in conjunction with the smooth movements of the human eye across a page, thus improving comprehension [1]. However, due to the spontaneous saccadic movements our eyes make in every moment, it is difficult to ascertain the validity of such a claim. Some say that sans-serif fonts take out complexity of lettering, thus making it easier for the eye to naturally distinguish between letters. Others, however, claim that the complexity of serif fonts makes it easier for the eye not to perceive the letters, but rather to distinguish similar letters from each other (for instance, when using a sans-serif font, a lower-cased L and an upper cased I look identical, whereas serif fonts allow the eye to distinguish between the two characters) [2]. Because reading is a fundamental part of our everyday lives, diving into some of the specificities of how we actually read can be intriguing. Does the style of font actually affect our ability to comprehend reading material? Could it also affect the speed at which we read? These are the two fundamental questions our work intends to answer. Furthermore, using the Gazepoint GP3 eye tracker, we will not only be able to attempt to answer these questions, but also obtain insight into the potential differences in eye movements that naturally may occur when reading passages written in different fonts. John Welter Clemson University jcwelte@clemson.edu People often prefer to use one font style over the other. Some, for instance, enjoy reading articles printed in sans-serif fonts because they seem easier on the eyes to me due to their simplicity. On the contrary, others may prefer serif fonts because they are aesthetically pleasing over long periods of time. Because individuals may have these preferences, preference may play a key role in the potential difference in reading performance. Nevertheless, our hypothesis is that neither serif fonts nor sans-serif fonts have any pronounced difference on performance. 2 Background Many studies have been done regarding eye movement and reading, such as the effect of reading out loud on fixations [3], and the comprehension of reading annotated passages [4]. While they studied various reading strategies, they never studied the effect the text style had on the reading itself. Cheng et. al came close with the annotated text, guiding reading with shaded boxes. Research has been done in regards to serif v. sans-serif study, but many methods and papers have been flawed [5, 6, 7]. One study determined that more than the font, things like size and color contrast have a greater effect on legibility [8]. However, there has never been any evidence regarding which type of font is better. In our experiment, we intend to observe for ourselves the potential impact. 3 Methodology Apparatus For this work, we used the Gazepoint GP3 Eye Tracker, a classic pupil/corneal reflection tracking device. The GP3 was mounted to a 22 DELL P2213t monitor. Gazepoint, the eye tracker manufacturer, claims accuracy within 0.5 to 1.0 degrees, and reads data at a rate of 60Hz. Participants were seated at the desk and asked to move roughly 15-20 inches away from the monitor. Because this study is not primarily focused on exactly where participants are looking, and rather objectively how long they view a certain general area in the text (to determine at what point they finish reading the text and get a general idea of reading speed across the articles), the use of a chin rest or other such restraint was unnecessary. The eye tracker was calibrated using the default 5-point calibration test using Gazepoint s included software. Stimuli The stimuli for our experiment are 4 images of two spate articles, written in both serif an sans serif fonts. Both articles come from Wikipedia. Article A describes a laughter epidemic in Uganda, while Article B describes the informal Waffle House Index. The articles were chosen based on the assumption that they are not general knowledge, and would therefore lessen the chance of a participant having prior knowledge of the subjects. Furthermore, both articles are easy to understand. The serif images are written in 29 pt. Times New Roman, while the sans serif images are written in 27.5 pt. Arial.
Figure 1 Group 1 s stimuli: serif article A, sans-serif article B Figure 2 Group 2 s stimuli: sans-serif article A, serif article B
Subjects To ensure some consistency with the experiment itself and to prevent large variances that naturally occur from person to person, we opted to do use a within subjects approach. Our primary subjects were university students at Clemson University. In our study, we have two groups of participants; participants in one group receive a sans-serif version of article A and a serif version of article B, and participants in the other group receive a serif version of article A and a sans-serif version of article B. Each group is composed of 10 participants. Subjects were Clemson University students between the ages of 19 and 29. plugin, we generated data comparing the average total reading speed of participants between sans-serif and serif articles. Figures 3.1 and 3.2 summarize the overall means. Experimental Design The design of this experiment follows a 2x2 pattern, since there are two versions of two different stimuli. To reduce variance between each individual and to increase reliability, we opted to test within subjects rather than between subjects; each participant will read an article displayed in a serif font followed by an article in sans-serif font, or vice versa. Each participant uses the Gazepoint GP3 system for the study. For the purposes of gathering line-by-line data regarding the time each participant spends reading, each line of each article is an independent area of interest (AOI). Figure 3.1 Comparison of times between conditions of article A Because we are not so concerned with pinpoint accuracy with the eye trackers and instead use them to get a general idea of where our participants are looking at any given time, devices such as chin rests to restrain stray head movement are not used. After each article, participants are given short comprehension quizzes consisting of four multiple choice questions regarding the contents of the article. The questions cover specific information such as dates or names, as well as more general facts. Procedures After greeting the participant and gathering demographic information, we seat him in front of the testing monitor. We then guide him through the 5 point calibration process. Next, we expose him to the first stimulus. The participant may read at his own pace without a time limit. After he finishes reading, he is given a short quiz on the material read. Afterwards, the eye tracker is recalibrated, and the participant is exposed to the second stimulus. After reading the second stimulus, the participant asked to complete a second quiz. He is thanked for his time and dismissed. 4 Results In total, participants performed 2*2*10 = 40 trials, of which all analyzed. Using the CSV files exported from Gazepoint and the graded test results, we were able to gather quantitative data regarding reading speed and comprehension. Results were calculated using a 90% confidence interval for the population mean. Scan paths were created for qualitative analysis by proprietary functions in Gazepoint. Figure 3.2 Comparison of times between conditions of article B Ultimately, these results are evidence that there is no correlating effect on reading speed between sans-serif and serif fonts (Article A: P=0.453; Article B: P=0.295). Reading Comprehension In order to gauge the effect on reading comprehension, it was first necessary to calculate the average score of each participant on each of their tests. Using Minitab 17 statistical analysis software s ANOVA One-Way plugin, we generated data comparing the average total score of participants between sans-serif and serif articles. Figures 4.1 and 4.2 summarize the overall means. Reading Speed In order to gauge the effect on reading speed, it was first necessary to calculate the total time each participant spent reading the article. In order to find a participant s total reading time, the total times spent in each AOI were added together. Using Minitab 17 statistical analysis software s ANOVA One-Way
Overall, due to the incredibly high P-values and margin of error for our means, we believe this is evidence that there is little, if any, difference between comprehension and/or reading speed when using a serif or sans-serif font. Figure 4.1 Comparison of test scores between conditions of article A Figure 5.1 Scan path of a participant in group B who scored a 75 on this article s quiz. Notice how the first half of the article is seemingly ignored. (Reading time: 13s) Figure 4.2 Comparison of test scores between conditions of article B Similar to the speed comparisons, these results seem to imply that there is no correlating effect on reading speed between sansserif and serif fonts (Article A: P=0.211; Article B: P=0.691). 5 Discussion Reading Speed vs. Comprehension results It is interesting to note what appears to be a strange and contradictory correlation between the results of the reading speed analysis and the comprehension analysis within our sample. Where one would believe a longer reading time would denote a higher score, since it indicates more focused reading, the results within our sample happened to be the exact opposite. In figure 5.1 and 5.2, we show two scan paths from group B. The first shows a reader who skimmed the articles and made an average score of 75 on both tests, while the second shows a reader who read the article twice over and made an average score of 25 on both tests. Furthermore, average reading speed within our sample seemed to be entirely unrelated to font style, since the average reading time taken seemingly differs between articles A and B. While these certainly do not have any lasting value (since the P-values are so high), they do help point to the null hypothesis that it doesn t matter which font style is used. Figure 5.2 2-part scan path of a participant in group B who scored a 25 on this article s quiz. Notice how they look back for relevant information. (Reading time: 60s) Reading Speed vs. Scan Paths Another interesting note is the small difference in the shape, spacing, and fixations-per-line of the scan paths between participants. Though some scan paths varied greatly from others (such as the examples in figure 5), scan paths mostly followed the
same general figure. Figure 6.1 and 6.2 demonstrate this, showing two participants scan paths whose reading times have a difference of greater than 20 seconds. experiment could involve testing how a speed reader s eyes move across a page as opposed to a person who simply skims a passage. The gaze data produced from this experiment could also be used as a means of modeling eye movements for use in creating more realistic 3D animations. References 1. De Lange, R. W., Esterhuizen, H. L., Beatty, D. (1993). Performance differences between Times and Helvetica in a reading task.electronic Publishing, 6(3), 241-248. 2. Reynolds, L. (1979). Legibility studies: Their relevance to present-day documentation methods. Journal of Documentation, 35(4), 307-340. Figure 6.1 a fast reading participant s scan path 3. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological bulletin, 124(3), 372. 4. Shiwei Cheng, Zhiqiang Sun, Lingyun Sun, Kirsten Yee, and Anind K. Dey. 2015. Gaze-Based Annotations for Reading Comprehension. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1569-1572. DOI=10.1145/2702123.2702271 5. Tinker, M.A. (1963). Legibility of Print, 3rd edition. Iowa: Iowa State University Press. 6. Burt, C. (1959). A psychological study of typography. Cambridge: Cambridge University Press.PAKE, F. I., AND WATERS, K. 1996. Computer Facial Animation. A. K. Peters. Figure 6.2 a slow reading participant s scan path Possible improvements Due to a small sample size, short questionnaire, and only testing one font per condition, we believe that the experiment could be improved upon to create more telling, concrete results. More participants would significantly narrow down the margin of error of the results and would be ideal, as well as having a more comprehensive questionnaire with a wider range of potential scores. The Gazepoint tracker itself is good for basic analysis, but more precise tracking devices could provide deeper insight into the possible effect of fonts on eye movement and fixations, since 1 degree is often enough to shift the assumed focal point of the participant up or down an entire line of text. 7. Robinson, D.O, Abbamonte, M., Evans, S.H. (1971). Why serifs are important: the perception of small print.visible Language, 4, 353-359. 8. Bix, L. (2002). The Elements of Text and Message Design and Their Impact on Message Legibility: A Literature Review. Journal of Design Communication, No. 4. 9. "Tanganyika Laughter Epidemic." Wikipedia. Wikimedia Foundation. Web. 1 Oct. 2015. 10. "Waffle House Index." Wikipedia. Wikimedia Foundation. Web. 1 Oct. 2015. Future Work While the effect of serifs on reading is still hard to determine, the idea of reading speed and comprehension is a wide topic with many possible testable variables. For instance, a possible