Most Misleading Graphs

Transcription

1 Most Misleading Graphs The road to hell is paved with bad graphic designers! This image, from the cover of the Ithaca Times (Dec. 7, 2000) may well be the most misleading graph ever published. It provides a spectacular example of more graphical sins than I have ever seen in one image. The cover story, "Why does college have to cost so much?" shows a large graph superimposed on a scene from the Cornell campus. There are two jagged lines running across the graph, one labeled "Cornell's Tuition" and the other "Cornell's Ranking". The tuition graph shows a steady rise, and the ranking graph, after some early meandering, plummets to an all time low. The clear impression is that students are paying more for far less.. What s wrong with this graph? What s missing?

2 Page 2 More careful reading of the whole article (buried several pages into the paper) reveals a different story: (1) The ranking graph covers an 11 year period, the tuition graph 35 years, yet they are shown simultaneously (the same apparent width) on the same horizontal "scale". (2) The vertical scale for tuition and ranking could not possibly have common units, but the ranking graph is placed under the tuition graph creating the impression that cost exceeds quality. (3) The differing time units are cleverly disguised by printing them rotated 90 o. (4) And here is the masterstroke: the sharp "drop" in the ranking graph over the past few years actually represents the fact that Cornell's rank has IMPROVED from 15th TO 6th... [Credits: Dave Bock, from Ithaca, NY provided these graphs, and most of the accompanying text. He says: "Egad. There must be an award this newspaper deserves." Yes, and now they've gotten it!]

3 Page 3 The Challenger Disaster The Space Shuttle Challenger exploded shortly after take-off in January Subsequent investigation determined that the cause was failure of the O-ring seals used to isolate the fuel supply from burning gases. This figure (scanned badly from Wainer, 1995) shows a graph accompanying the Report of the Presidential Commission on the Space Shuttle Challenger Accident, 1986 (vol 1, p. 145) in the aftermath of the disaster. NASA staff had analyzed the data on the relation between ambient temperature and number of O-ring failures (out of 6), but they had excluded observations where no O- rings failed, believing that they were uninformative. Unfortunately, those observations had occurred when the launch temperature was relatively warm (65-80 degf). Reanalysis of the O-ring data involved fitting a logistic regression model. This provides a predicted extrapolation (black curve) of the probability of failure to the low (31 degf) temperature at the time of the launch and confidence bands on that extrapolation (red curves). See also Tappin, L. (1994). "Analyzing data relating to the Challenger disaster". Mathematics Teacher, 87, There's not much data at low temperatures (the confidence band is quite wide), but the predicted probability of failure is uncomfortably high. Would you take a ride on Challenger when the weather is cold?

4 Page 4 Rubber-band Scales. (from Wainer, 1997, p.29) A less obvious (and therefore more insidious) way to create a false impression is to change scales part way through an axis. This graph, originally from the Washington Post purports to compare the income of doctors to other professionals from (graph on the left) It surely conveys the impression that doctors incomes increased about linearly, with some slowing down in the later years. But, the years have large gaps at the beginning, and go to yearly values at the end. If you are going to use rubber-band scales, try to make the axis values small and visually indistinct, as this graphic sinner did! A re-drawing of the graph, with year values spaced appropriately (graph on the left), gives an entirely different message -- doctors income now appears to have increased exponentially, while the incomes of other professionals seems to have increased linearly. The annotation of the date when Medicare began suggests another interpretation -- linear increase, but with two different slopes before and after the implementation of the Medicare system. We can think about these competing models of the data now that the data are portrayed fairly.

5 Page 5 The Lie Factor: Ed Tufte of Yale University has defined a lie factor as a measure of the amount of distortion in a graph. Lie Factor = size of effect shown in graphic/size of effect shown in data If the lie factor of a graph is greater than 1, the graph is exaggerating the size of the effect. The lie factor for this graph is approximately A much more accurate representation is shown below:

6 Page 6 Common Sources of Distortion The use of 3 dimensional effects is a common source of distortions in graphs. Another common source is the inappropriate use of linear scaling when using area or volume to represent values. Lie Factor = 9.4 USA Today Graphs: