Who Wants Airbags? All passenger vehicles sold in the United States

Transcription

1 Government studies show that airbags save lives. A new study shows the opposite. Meyer and Finney describe their analysis and explains why it reaches a different conclusion. Who Wants Airbags? Mary C. Meyer and Tremika Finney Airbag Controversy All passenger vehicles sold in the United States of model year 1998 and later are required by law to have airbags for both driver and front seat passenger. The National Highway Traffic Safety Administration (NHTSA) estimates that airbags installed in automobiles have saved more than 10,000 lives as of January NHTSA also confirms that 238 people have been killed by airbags in the years 1990 through 2003, and many airbagrelated injuries have been reported. But because 10,000 is a much larger number than 238, it is generally agreed that airbags are a good thing. Most of the controversy has centered around legislation: Should people have a choice about whether to have these devices in their cars, or should they be mandatory, like seatbelt use? Many people have objected to having airbags forced on them, enough to prompt NHTSA to allow for installation of on/off switches under special conditions. Information about the deaths attributed to airbags can be found on the NHTSA web site. Each death occurred in a low-speed crash, in circumstances in which it was clear there was no other logical cause of death. Is it reasonable to think that airbags can kill people only at low speeds? Perhaps airbags are responsible for some deaths at high speeds as well, but these deaths may be attributed to the collision instead of to the airbag device. AP Photo/Insurance Institute for Highway Safety CHANCE 3

2 Musings Clues in a Bent Steering Wheel NYT Jan. 18, 2005 Emergency room doctors who want to learn quickly whether a crash victim suffered serious chest or abdominal injuries should find out what the steering wheel looks like. The more bent the steering wheel, the more likely the patient is to have a thoracic injury, a new study reports. Writing in Annals of Emergency Medicine, the researchers... said that rescue workers should be encouraged to look at the steering wheel and report damage to the doctors.... The study drew on data generated over eight years from 42,860 people who were in the front seat during crashes. The researchers found that every steering wheel dent of roughly two inches increased the driver s risk of chest injury by 28 percent. It was not clear whether the wheel was causing the injuries or was simply a measure of the force and nature of the crash. One way to estimate the number of lives saved or lost is a result of a simple calculation. Take all of the highway crashes in a given time period and compare the outcomes for occupants with airbags to those for occupants without airbags you will find that the proportion of people killed is smaller for the airbag group. Take the proportion of people killed in the no-airbag group and apply it to the occupants in the airbag group to get the number of people who would have been killed, if there were no airbags. The difference between this number and the actual number killed is the number of lives saved. This calculation is accurate if the drivers and passengers with airbags behave similarly to those in cars without airbags, and if the crashes occur under the same conditions. However, if occupants with airbags tend to be more safetyconscious people, the comparison might not be fair; other factors might be contributing to the difference in probability of death, such as how fast the car is moving and whether the occupants are wearing seatbelts. In this article, we attempt to control for possible confounding factors to get an accurate assessment of the effectiveness of airbags. The data used come from the NHTSA web site and are available to the public. We also attempt to assess the effectiveness of airbags for various groups such as the elderly, passengers versus drivers, occupants in smaller versus larger vehicles, and women versus men. Who should have airbags and who should turn them off? Is there enough evidence to justify a federal mandate of airbags for everyone, rather than an option for the individual? Assessing Lives Saved NHTSA compiles data from automobile crashes on a yearly basis, using a random sampling method. The National Center for Statistics and Analyses (NCSA), a division of NHTSA, has developed the National Automotive Sampling System (NASS) Crashworthiness Data System (CDS). All police-reported crashes in which there is a harmful event (personal or property), and from which at least one vehicle is towed, are considered for inclusion into the dataset. The random selection of crashes is stratified, to ensure that a representative sample is collected. The country is split into primary sampling units (PSU), which are stratified into types of geographic regions such as city, suburb, rural areas. Within each PSU stratum, two PSUs are randomly selected. Within each PSU, the sampling procedure is stratified according to several variables: the severity of the crash in terms of the greatest injury to one of the occupants and whether the vehicles are towed. The more serious accidents are intentionally oversampled. More details found at the author s web site, edu/~mmeyer/airbags.htm, and at the NHTSA web site, Because some types of crashes are oversampled, the database has weights to reflect the actual proportions of the types of crashes in each strata. These weights can be used when analyzing the data to get accurate percentages of fatalities as well as to compute standard errors and confidence intervals for analysis results. The NHTSA third report to Congress refers to this database as the most comprehensive, representative crash investigation system available and has the most accurate safetybelt use reporting of any file available to NHTSA. The CDS system has recorded information about airbags in a consistent manner since The proportion of occupants with airbags was small in the early years, but is steadily increasing, with more than one-third of drivers in the CDS database having airbags by A few years hence, only occupants in old beaters will not have airbags available. In the earlier years of the 1990s, airbags were an option found more often in more expensive cars, and more often on the driver s side. From 1997 to 2002, airbags were neither usual nor unusual; hence, these are the years chosen for analysis. The data are available to the general public, and anyone with a computer and software can download them and reproduce all the analyses shown in this article. We keep records about whether the occupant is 16 years or older and sitting either in the driver s seat or in the front passenger s 4 VOL. 18, NO. 2, 2005

3 Table 1 Comparison of weighted fatality rates for occupants with airbags available with those for occupants without airbags available. The weighted percentages more accurately reflect the population proportion of fatalities Airbag No Airbag Number Killed Total Occupants 5,742,815 4,854,906 % Killed 0.34% 0.58% seat. We delete records for occupants in the backseat and center front, as well as occupants sitting in laps or other nonstandard positions. Further, we delete records with incomplete information for important variables such as impact speed, direction of force, seatbelt use, etc. There are complete records for 22,804 occupants; using the weights, we estimate that these records represent 10,597,721 front-seat occupants over the age of 16 who were involved in highway crashes from 1997 to A substantial proportion of observations were deleted because of missing data (almost half). We are assuming that the complete observations still represent a random sample of the nation s accidents. If cars with airbags, for example, are more likely to have missing data, this throws some doubt on the results presented here. However, the dataset is still the best available for these analyses. About 4 percent of the 22,804 occupants died in the crash or afterwards, as a result of injuries sustained. However, because the more severe crashes were oversampled, this 4 percent is an inflated estimate of the probability of dying in any police-reported crash with at least one towed vehicle. Applying the weights, we get only about 0.6 percent of the represented occupants died. For all of the following analyses, the data are weighted. The most straightforward assessment of airbag effectiveness can be illustrated by a simple table. Suppose you have an airbag in your car. If you crash, are you really less likely to be killed than if you don t have one? For this comparison, we won t worry about whether the airbag deployed, or the cause of death, etc. We simply compare the proportion of deaths for occupants with airbags to the proportion of deaths for occupants without airbags. For the data, about 56 percent of drivers had airbags available, compared with 47 percent of passengers. We see in Table 1 that 0.34 percent of occupants with airbags available were killed, compared to 0.58 percent of occupants in cars without airbags, indicating that airbags are associated with a 41 percent reduction in the probability of death. Let s estimate the number of lives saved by airbags using the simple formula. If we apply the higher fatality rate to the 4,854,906 people in cars with airbags, we get 33,031 people killed, instead of the actual 19,276 fatalities in this category. This simple calculation suggests that for the occupants represented by this dataset, approximately 13,755 lives were saved due to airbags. A common way to assess the statistical significance of an effect is the odds ratio. The odds of an event, such as being killed, are defined as the probability of the event divided by the probability of the event not happening. The odds can be any positive number, and odds greater than one means the event is more likely to happen than not. The odds ratio is associated with a predictor of the event. For example, the odds ratio for airbags is the odds of death with airbags divided by the odds of death without an airbag. We also can compute a 95 percent confidence interval for the odds ratio to assess statistical significance. The odds ratio can be computed from the information in Table 1; the confidence interval was calculated in a statistical software package called SUDAAN, which specializes in complicated study designs such as our stratified, weighted sampling. Effect Odds Ratio 95% CI Airbag 0.58 (.48,.71) The odds ratio is interpreted as: The odds of death with an airbag are 58 percent of the odds of death without an airbag. If the odds ratio were exactly one, the odds of death with and without an airbag would be exactly the same or, airbags would have no effect on the probability of death. Note that the confidence interval for the odds ratio does not contain one, so that we say the effect of airbags is statistically significant. How do airbags compare with seatbelts, in terms of saving lives? We can make the same table for seatbelts. Here, seatbelt means that the occupant had a seatbelt and used it properly. No seatbelt refers to any other situation, including no seatbelt available, seatbelt available and not used, as well as when the seatbelt is not used properly, such as Graphic from The ABCs of air bags (U.S. Department of Transportation, National Highway Traffic Safety Administration) CHANCE 5

4 Table 2 Comparison of fatality rates for occupants with using seatbelt properly with those for occupants not using seatbelt properly Seatbelt No Seatbelt Number Killed 16,001 31,199 Total Occupants 7,774,635 2,823,086 % Killed 0.21% 1.11% putting the shoulder behind the back. For the data, we have 73 percent of the occupants using seatbelts properly. Table 2 shows the fatality rates by proper seatbelt usage. The difference in percentages killed with or without seatbelts is larger than the difference in percentages killed with or without airbags, indicating that seatbelts are more effective. Our simple calculation indicates that about 85,920 of the 7,774,635 people wearing seatbelts would have been killed if the fatality rate was 1.11 percent, so that about 69,919 lives were saved due to proper use of seatbelts. The effect of a seatbelt on the probability of death is statistically significant, as seen below. The odds ratio is small and the 95 percent confidence interval is far from one. The odds of death for occupants properly wearing seatbelts is only about 18 percent of the odds of death for occupants who are not properly wearing seatbelts; in other words, the odds of dying are reduced by 82 percent for seatbelt wearers. of airbags is really due to increased seatbelt use. We see in Table 3 that occupants with airbags available are actually much more likely to be wearing seatbelts properly (almost 85 percent compared with less than 60 percent). There might be several reasons for this. First, in the earlier years when air bags were an option, the more safety-conscious people, the seatbelt wearers, would be more likely to have them. Second, both airbag availability and seatbelt use have been increasing over the years. Third, people of higher socioeconomic status (such as those who might have newer cars) tend to make better health decisions (such as wearing seatbelts). In any case, we need to ask: If people in cars with airbags are more likely to be wearing seatbelts, then how much of the improvement in fatality rates with airbags is really due to seatbelts? The numbers for occupants and fatalities are shown for each seatbelt/airbag combination in Table 4. We see that the fatality rate for people with airbags is reduced if they re wearing their seatbelts. However, if the occupant is not properly belted, there is a slight increase in fatality rate if the airbag is available. We can calculate the number of lives saved due to airbags, taking into account whether occupants are wearing seatbelts. For the seatbelt wearers, there were 8,626 deaths out of the 4,871,940 occupants in the airbag available group. If the death rate were equal to the no-airbag rate, there would have been 12,377 deaths, so we conjecture that 3,751 lives Table 3 Comparison of rates of proper seatbelt usage for occupants with airbags available with those for occupants without airbags available Effect Odds Ratio 95 % CI Seatbelt 0.18 (.13,.27) Overreporting of seatbelt use in low-severity crashes is a known problem in estimating seatbelt effectiveness. Because many insurers stipulate the use of the seatbelt, an occupant who is able to move around immediately after the crash might report to the police that he or she had been wearing the seatbelt, even if this is false. The NASS CDS database attempts to determine safety use by the preponderance of evidence obtained from the inspection of the vehicles involved but evidence might be absent in low-severity crashes. We take the seatbelt effectiveness numbers with a grain of salt. Airbag and Seatbelt Confounding To get a more accurate picture of lives saved by airbags, we need to account for the seatbelt effect. If people in cars providing airbags are more likely to be wearing their seatbelts, then perhaps some of the apparent effectiveness Airbag No Airbag Seatbelt 4,871,940 2,902,694 No Seatbelt 870,875 1,952,211 % Wearing Seatbelt 84.8% 59.8% Table 4 Comparison of fatality rates for airbag/seatbelt combinations Seatbelt No Seatbelt Airbag =. % =. % No Airbag =. % =. % VOL. 18, NO. 2, 2005

5 Figure 1. Percentages of deaths by speed of impact, seatbelt use, and airbag availability. The plot on the right is a magnification showing the lowest speed categories. were saved. However, in the no-seatbelt group, we have a higher proportion of deaths in the airbag group. Applying the same formula, we get 1,483 lives lost due to airbags for occupants who weren t wearing seatbelts, for a grand total of 2,268 lives saved. Note that this is considerably smaller than the initial estimate of 13,755 lives saved. The odds ratios associated with airbags and seatbelts, modeled together, are shown below with confidence intervals. The third row describes the interaction between seatbelt use and airbag availability. Because the confidence interval for the interaction effect contains one, we say the interaction is not significant; that is, the effect of the airbag on the probability of death does not depend on whether or not the seatbelt is used properly. In this table, we see that the only statistically significant effect is that for seatbelts, and that seatbelts very significantly reduce the probability of death in a crash. Effect Odds Ratio 95 % CI Seatbelt 0.24 (.13,.45) Airbag 1.16 (.81,1.66) Seatbelt * Airbag 0.60 (.28,1.26) We interpret the odds ratios as follows. If you don t have an airbag, then the odds of death if you properly use your seatbelt are 24 percent of the odds of death if you don t use your seatbelt properly. On the other hand, if you have an airbag, the odds of death if you properly use your seatbelt are.24 x.60 = 14.4 percent of the odds of death if you don t use your seatbelt properly. If you are not using your seatbelt properly, the odds of death are 16 percent higher with an airbag. Note that the confidence intervals for the second two odds ratios contain one, so that the airbag effect is not statistically significant. The first calculation showed a significant beneficial airbag effect, but when we take into account seatbelt use, in particular noting that occupants with airbags available are more likely to wear seatbelts, we get a different picture. This is an example of a confounding factor we get a different conclusion when we account for another variable in the model. The new assessment of airbag effectiveness is more accurate because it controls for a confounding variable, but maybe the picture is not yet complete. We know that other factors, such as impact speed, affect the probability of death. If people who have airbags available are more reckless and drive faster, then impact speed may be a confounding factor. If the airbag group were crashing at higher speeds, this would inflate the percent of deaths in this group and result in an unfair comparison. We need to take into account other variables that affect probability of death to get a more accurate picture. Impact Speed, Seatbelts, and Airbags The measure of impact speed is called total delta-v in the dataset. Specifically, delta-v is defined as the vector velocity change during the collision phase of the crash. The NHTSA dataset records both latitudinal and longitudinal delta-v, as well as total delta-v, which is used here. For some records, the total delta-v is accurately inferred from data relating to the crash. For other records, the delta-v is estimated to be within ranges: 0 9 kilometers per hour (kph), 10 24, 25 39, 40 54, and 55+; these categories will be used in the analyses. The sample sizes for each impact speed category are 599, 11,337, 7,235, 2,462, and 1,171, respectively. The weighted percentages of deaths in the database are shown in Figure 1 by delta-v category, seatbelt usage, and airbag availability. The percentage of deaths in each combination is represented by a plot character; the connecting CHANCE 7

6 Table 5 Fatality rates by impact speed level, airbag availability, and seatbelt use, as reflected in Figure 1 Seatbelts Used Properly Seatbelts Not Used Properly Delta-V Airbag Group No-Airbag Group Lives Saved Airbag Group No-Airbag Group Lives Saved kph = 0.0% kph = 0.12% kph = 1.28% kph = 3.91% kph = 16.55% Totals: lines are for ease of presentation. The probability of death increases with increasing delta-v, more steeply for higher speeds. We also can see a distinct difference in probability of death according to proper seatbelt use: the two lower lines represent the proper seatbelt use categories. The plot on the right is a close up of the lower-impact speed categories. We see that the airbag-only group seems to have a substantial proportion of deaths at lowest speeds, compared to the other groups. We will check to see if this effect is statistically significant. The calculation of the number of lives saved by airbags is getting more complicated. We compute the number Table 6 Odds ratios by impact speed level, airbag availability, and seatbelt use for significant effects only, plus the main effect for airbag. The interaction terms were tested and are not included if not significant Effect Odds Ratio 95% CI Seatbelt 0.33 (.23,.47) Airbag 1.19 (0.97,1.46) 0-9 kph 0.69 (.25,1.92) kph 1.00 (1.00,1.00) kph 13.6 (9.7,18.9) kph 58.8 (35.8,96.4) 55+ kph 329 (195,554) Airbag * no seatbelt * 0-24 kph 3.73 (1.77,7.88) of lives saved in each category, as shown in Table 5. For example, let s look at the kph impact speeds and seatbelt wearers. The 0.37 percent deaths in the no-airbag group are applied to the 785,646 people with seatbelts and airbags, to get a projected 2,896 deaths. There were in fact 2,247 deaths, so that the estimated number of lives saved in this category is 649. However, the percentages of death are higher in the airbag group for other categories, so that totaling over all categories, we get 3,727 lives lost due to airbags. The results for airbags are getting worse! The explanation for the confounding effect of delta-v on the relationship between airbags and the probability of death is as follows: Occupants with airbags crash at significantly lower speeds, on average, than occupants without airbags. In fact, about 3.5 percent of the airbag group crashed at the two highestimpact speed categories, compared with about 6.5 percent of the no-airbag group. The same pattern holds for seatbelts. The estimate of lives saved due to seatbelts will be lower once the impact speed is included in the calculation. However, the overall effect of seatbelts is still to save many lives. When we account for delta-v (and airbags), our estimate of lives saved due to seatbelts is now only 42,056, rather than the previous 85,920. These effects are summarized conveniently in terms of odds ratios in Table 6. The first row tells us that a seatbelt reduces the odds of death by 67 percent for any given speed category and airbag availability. The main effect of airbags is not statistically significant; the odds ratio confidence interval contains a one, indicating that there is no effect of airbags on the whole. The odds ratios for the given speed categories are compared with the reference category of kph. Thus, the odds of dying at 0 9 kph are about 69 percent of the odds of dying at kph; similarly, the odds of dying at kph are almost 14 times higher than the odds of dying at kph. The last line of the table represents a significant interaction between low speed and airbags. For unseatbelted occupants in collisions with impact speed up to 24 kph, the odds of dying are almost 8 VOL. 18, NO. 2, 2005

7 Figure 2. Percentages of deaths by speed of impact, seatbelt use, and airbag availability for frontal and nonfrontal collisions. four times higher with an airbag than without. This effect was seen in the close-up version of the plot of percent death against delta-v. The interactions between the other effects were tested and found to be not significant, and so were dropped from the model and not presented. This means, for example, that the effect of airbags on the probability of death does not significantly depend on the seatbelt use. These analyses show that airbags are actually detrimental at low speeds and not helpful at higher speeds. Can this be correct? What about other possible confounding factors? Let s try to include as many significant effects as possible before we draw a conclusion about airbags. For the rest of this section, we examine effects of other variables through plots, then build a comprehensive model for the probability of death in a collision, using as many significant predictors as we can find. Comparing Frontal and Nonfrontal Collisions Airbags are designed to be effective in frontal collisions, so our analysis needs to account for direction of impact as well as speed of impact. Figure 2 shows proportions of deaths by our seatbelt/airbag categories as before, but separating frontal and nonfrontal collisions. We see that the percent of deaths in the nonfrontal collisions are consistently higher across groups. Most of the nonfrontal collisions take place at lower speeds, so the erratic pattern in the higher speed categories might be due to random variation in small sample sizes. Figure 3 shows the same thing, but a close up of the first three impact speed categories. We see the higher probability of death in the airbag-only group for the lower speed categories, seen in the previous plots, is accounted Musings Moral Hazard on the Road The Atlantic, Jan/Feb 2005 To the roll of highway perils drunks, cell phones, teenagers can be added one more: the insured. A new study suggests that requiring insurance coverage may lead drivers to take a cavalier attitude toward road safety. Economists from Columbia University and Harvard Law School s Olin Center for Law, Economics, and Business examined the effects of the compulsory-insurance laws and no-fault insurance systems that were gradually adopted in the United States from 1970 to (Under no-fault systems each driver s insurance pays for the damages that he or she inflicts, though all states give drivers some exposure to lawsuits for negligence. Under older, fault-based systems, the at-fault driver pays all the damages.) Forty-five states now have compulsory insurance, and 14 have no-fault systems. The researchers found that relaxation of liability is correlated with a 10 percent increase in traffic deaths (or about 4,000 dead travelers a year). Uninsured drivers tend to drive more carefully (after all, they, themselves, have to pay for accidents): for every 1 percent decrease in the number of uninsured drivers, the number of fatalities increases by 2 percent. Despite the moral hazard posed by insurance, the authors concede that compulsory and no-fault systems aren t all bad: though more people may die, the victims families are far better remunerated than in the past. CHANCE 9

8 Figure 3. Percentages of deaths by speed of impact, seatbelt use, and airbag availability for frontal and nonfrontal collisions, first three speed categories. Figure 4. Percentages of deaths by speed of impact, seatbelt use, and airbag availability for older and younger occupants. for mostly in the nonfrontal collisions. Perhaps in frontal collisions, airbags do not have a detrimental effect at low speeds. However, we see that the fatality rates are higher for the airbag groups at the highest speed level. Older Occupants People age 55 and older might be more frail than other occupants, and more likely to be killed in an accident, all other things being equal. Do airbags pose a special risk to older occupants? We reproduce the plots of percent deaths by impact speed category and seatbelt/airbag groups, comparing younger and older occupants in Figure 4. The most obvious finding is that older people are more likely to be killed, and at the highest speed level the airbags seem to be detrimental to older occupants. Comparing Driver and Passenger Safety Is there any difference between driver s side and passenger s side airbags? It seems reasonable that passengers are more 10 VOL. 18, NO. 2, 2005

9 likely to be out of place, which can be dangerous when the airbag deploys. If the passenger is slouching against the window, or reading, or leaned back asleep, this can cause serious injuries when the airbag deploys. On the other hand, the presence of the steering wheel might pose a danger for drivers, as the airbag is closer to the driver when it deploys. Figure 5 shows proportions of deaths for drivers and passengers, in each speed category and airbag/seatbelt combination. There are more drivers than passengers (80 percent versus 20 percent), and interestingly, cars with passengers crash at lower speeds, on average, so the highest speed category for passengers has a small sample size. For drivers, the airbag-only group has the highest proportion of deaths, except in the middle speed category. The passenger plot shows a detrimental effect of airbags in both seatbelt and no-seatbelt groups; we will test to see if this is statistically significant when all other variables are controlled for, when we construct the comprehensive model. What about Car Size? We ll consider large vehicles to mean SUVs, pickup trucks, minivans, and regular vans, while smaller vehicles include all types of passenger cars, such as sedans, hatchbacks, station wagons, and convertibles. Figure 6 shows that, for smaller vehicles, the same patterns observed earlier are evident. We also can see that occupants in larger vehicles tend to be safer than those in smaller vehicles, Figure 5. Percentages of deaths by speed of impact, seatbelt use, and airbag availability for drivers and passengers. Figure 6. Percentages of deaths by size of vehicle, speed of impact, seatbelt use, and airbag availability. CHANCE 11

10 Figure 7. Percentages of deaths by speed of impact, seatbelt use, and airbag availability for men and women. especially if seatbelts are used properly. The airbag-only group has unusually high percent deaths for lower speeds, but this might be due to random variation. We postpone conclusions about the statistical significance of these observations until we formulate the comprehensive model at the end of this section. Table 7 Odds ratios and confidence intervals for the comprehensive model Effect Odds Ratio 95% CI Seatbelt 0.34 (.25,.45) 0-9 kph 0.50 (.16,1.6) kph 1.00 (1.00,1.00) kph 18.4 (12.5,26.7) kph 41.3 (21.6,79.2) 55+ kph 331 (183,599) Frontal 1.00 (1.00,1.00) Back 0.17 (.03,.93) Occupant side 6.36 (4.68,8.63) Other side 1.93 (1.27,2.94) Older 5.11 (3.19,8.19) Small vehicle * highspeed 2.53 (1.66,3.87) Airbag * no seatbelt * 0-24 kph 4.76 (2.76,8.23) Women and Men: Gender Gap? The original federal mandate required airbags to protect an unbelted average-sized male in a 35 mph crash. Subsequently, most of the people killed by airbags were children and smaller women. Passed in 1998, the Transportation Equity Act requires automakers to construct airbags to protect all sizes of occupants. The comparison of the percent of death by gender is shown in Figure 7 for the impact speed, seatbelt, and airbag combinations. Again, this comparison does not take into account the different effects of driver and passenger. Here, there is potential for confounding; the data show that women are more likely than men to be passengers, so that some of the difference in the plot might be due to the driver/passenger effect. The comprehensive model in the next section will allow us to separate the effects and determine which are important in estimating the probability of death. We notice that the seatbelt effect seems to be stronger for women than for men, but that in general, the patterns observed earlier are seen in both plots: percent death increases with total delta-v, and is smaller when seatbelts are worn properly. A Comprehensive Model The logistic regression model allows us to estimate a probability function, given many predictor variables. We can sort out effects of confounding and interactions between predictor variables, and also determine which are statistically significant. The results are presented as odds ratios with 95 percent confidence intervals to assess statistical significance. This will help give the clearest picture possible. When there are many predictor variables and many possibilities for interaction and confounding effects, the model-building procedure can be an extensive task. There 12 VOL. 18, NO. 2, 2005

11 are literally thousands of combinations of variables to consider. We used the following approach: we considered all main effects (predictors alone) and all two-way interaction terms. We also considered all three-way interaction terms that were suggested by the plots in the previous section, or by the literature, such as the claims in the NHTSA Reports to Congress. We choose the model with the smallest AIC (an information criterion that rewards a close fit but penalizes models with too many variables) and a good fit (Hosmer- Lemeshow lack-of-fit p-value should be greater than 0.05). The results for the final model are found in Table 7. All analyses were performed using SUDAAN. The seatbelt continues to be highly significant for saving lives in crashes. Also, as expected, the impact speed is a strong predictor of the probability of death. Four directions of force are considered. The nonfrontal group is split into three categories: collisions from the back, and side collisions either on the occupant side or the other side. For drivers, collisions from the left are occupant side, and for passengers, these are collisions from the right. We see that occupant side collisions significantly raise the probability of death: odds of death are estimated to be more than six times higher if the crash is on the occupant side, compared to a crash from the front. The odds of death for crashes on the other side are approximately twice as high for crashes from the front; rear collisions are safest in terms of fatality rates. We also see that older occupants have a significantly higher chance of being killed, and larger vehicles are significantly safer at higher speeds. Finally, the airbag significantly raises the probability of death for unseatbelted occupants in low-speed collisions (the same effect seen earlier still holds when other factors are accounted for). The main effect of airbag is not statistically significant, indicating that when all other factors are controlled for in the model, the only effect of airbag is to make lower speed collisions more dangerous. If the airbag main effect is included in the model, the odds ratio is 1.03 with 95 percent CI (.70, 1.51). Other effects that did not prove significant predictors are gender and driver versus passenger. In particular, there is no interaction between airbag availability and gender, indicating that airbags are not significantly more dangerous for women than for men. Also, we found that while older people have significantly higher risk of death, the interaction between airbags and age is not significant, indicating that there is no evidence that airbags pose a special risk to older people. Finally, the passenger/airbag interaction seen in Figure 5 proved not to be statistically significant. Airbag Deployment We have not considered deployment of the airbag to predict probability of death, only its existence. Ideally, an airbag should inflate only in frontal collisions at 25 kph or higher impact speed. Figure 8 shows airbag deployment rates for occupants with airbags available at each impact speed level. The percentage deployment is given for collisions where the main impact is in the front, side, or back of the vehicle. We see that the trend is as expected that is, airbags are more likely to deploy at higher speeds and in frontal collisions, but often deploy at low speeds or in side collisions, and sometimes also when the main impact is from the rear. For frontal collisions, it is difficult to use airbag deployment to Figure 8. Percent of observations for which the occupant s airbag deployed for frontal and nonfrontal collisions. Only observations for occupants with airbags are considered. CHANCE 13

12 Figure 9. Occupants in side collisions only. Proportions of fatalities are shown against three impact speed levels for combinations of seatbelt and airbag deployment. Not deployed includes cases in which there was an airbag that was not deployed and cases in which there was no airbag available. model the probability of death; the airbag almost always deploys in frontal collisions at high speeds, so the effects of speed and deployment are highly confounding. For side collisions, the airbag deploys only about half the time at higher speeds. Figure 9 summarizes the effects of seatbelt and airbag deployment combinations, split into occupant side collisions and other side collisions. Some impact speed categories are combined, due to small sample sizes in nonfrontal collisions in the 55+ kph group and the 0 9 kph group. We find a seatbelt-deployment interaction we get a detrimental effect of deployment when seatbelts are not worn. The logistic regression results for the subset of the data containing only side collisions are summarized in Table 8. The impact speed predictor is again highly significant, as is Table 8 Odds ratios for probability of death for occupants in side collisions Effect Odds Ratio 95% CI Seatbelt 0.38 (.19,.74) 0-24 kph 1.00 (1.00,1.00) kph 18.3 (12.5,26.7) 40+ kph 129 (57.3,291) Older 2.84 (1.57,5.16) Occupant side 3.17 (2.10,4.79) No seatbelt * deploy 3.21 (1.70,6.06) the seatbelt main effect. Again, we find that older occupants are more likely to be killed, and occupant side collisions are more dangerous than other side collisions. For side collisions, the size of the vehicle is not significantly associated with the probability of death. Finally, the combination of not wearing a seatbelt and airbag deployment increases the odds of death by about a factor of three. We have looked at the effectiveness of airbags on the probability of dying in an accident, controlling for other factors, and we have found that airbags are actually associated with increased probability of death in accidents. The situations in which airbags are most dangerous are low-speed collisions when the occupants are not wearing seatbelts. How can we reconcile our findings with previous studies that found airbags to be saving lives? For example, the study by Crandall, Olson, and Skiar (2001) is similar to ours, in that a logistic regression model is used to estimate odds ratios and the same effects are controlled for in the model. They looked at head-on collisions in passenger cars, and found the odds ratio for deployment of the airbags is 0.71, with 95 percent CI (.58,.87). This study used a different NHTSA database called the Fatality Analysis Reporting System (FARS). All accidents in which a fatality occurred are included in this database there is no random sampling procedure involved. All occupants involved in the crash are included, so that less than half of the occupants in the dataset actually died, although each was involved in a crash in which someone died. The methods outlined in the NHTSA reports to Congress also use the FARS database. The CDS dataset has two strata that represent fatal accidents. If we limit our CDS dataset to strata A and B, this can be considered a random sample of FARS data. Using 14 VOL. 18, NO. 2, 2005

13 Table 9 Odds ratios for probability of death for occupants involved in fatal collisions Effects Odds Ratio 95% CI Seatbelt 0.31 (.24,.41) 0-9 kph 1.14 (0.06,22.2) kph 1.00 (1.00,1.00) kph 1.89 (1.27,2.81) kph 1.12 (.43,2.88) 55+ kph 3.36 (1.77,6.38) Older 3.05 (2.42,3.85) Frontal 1.00 (1.00,1.00) Back 1.02 (.38,2.79) Occupant side 14.2 (9.74,20.8) Other side 1.50 (.96,2.35) No airbag 1.00 (1.00,1.00) Airbag, no deploy 1.28 (.67,2.43) Airbag, deployed 0.59 (.37,.95) Smaller vehicle * 40+ kph 4.26 (2.31,7.85) the same predictor variables, and using the variable selection procedure outlined above, we get the results shown in Table 9. The deployment of the airbag has an overall significant beneficial effect, in that deaths are reduced by about 41 percent after all other variables are controlled for. This result is similar to the Crandall result, but the confidence interval is larger because the sample size is smaller. How can we resolve these disparate results? When we look at the random sample of all accidents, we get that airbags are associated with increased risk of death, and this increase is due mostly to more deaths with airbags in low-speed crashes and no seatbelts. However, if we limit the dataset to include only collisions in which a fatality occurred, we get a significantly reduced risk of death due to airbags. This is analogous to doing a comparison with the same numerators but different denominators it is not unlikely that we will get different results. Here is a more dramatic analogy: If you look at people who have some types of cancer, you will see that those who get radiation treatment have a better chance of surviving than those who don t. However, radiation is inherently dangerous, and could actually cause cancer. If you give everyone radiation treatments whether they have cancer or not, you will probably find an increased risk of death in the general population. Making everyone have airbags and then verifying the effectiveness using only fatal crashes is like making everyone have radiation and then estimating lives saved by looking only at people who have cancer. Overall, there will be more deaths if everyone is given radiation, but in the cancer subset, radiation will be effective. Using the CDS dataset for the analysis, we find that airbags are associated with significantly more deaths, rather than fewer. Using a subset of the CDS data, we can mimic previous analysis and reproduce the results. Because of this, we are confident that our analyses better reflect the actual effectiveness of airbags in the general population. The evidence shows that airbags do more harm than good. References Third Report to Congress: Effectiveness of Occupant Protection Systems and Their Use. December National Highway Traffic Safety Administration, U.S. Department of Transportation, Washington, DC, Fifth/Sixth Report to Congress: Effectiveness of Occupant Protection Systems and Their Use. November National Highway Traffic Safety Administration, U.S. Department of Transportation, Washington, DC, Crandall, C.S.; Olson, L.M.; and Sklar, D.P Mortality reduction with air bag and seat belt use in head-on passenger car collisions. American Journal of Epidemiology 153(3): Agh, Dad! Musings The Simpsons Dec. 7, 1997 Lisa: Agh, Dad! Doesn t this car have seatbelts? Homer: Seatbelts, pff! They kill more people than they save! Lisa: That s not true, you re thinking of airbags! CHANCE 15

14 Discussion: Who Wants Airbags? Charles J. Kahane, PhD, National Center for Statistics and Analysis, National Highway Traffic Safety Administration I would not recommend using the Crashworthiness Data System (CDS) of the National Automotive Sampling System (NASS) for analyzing the fatality-reducing effectiveness of air bags, because a much larger data file the Fatality Analysis Reporting System (FARS), a census of the nation s traffic fatalities since 1975 is available for that purpose. The relatively small number of fatality cases on CDS and the high sampling error of statistics generated in this type of analysis make it difficult to obtain statistically meaningful, let alone precise estimates of fatality reduction. The best available estimates of fatality reduction by air bags are still the ones based on analyses of FARS, specifically the estimates in the National Highway Traffic Safety Administration s (NHTSA) Fifth/Sixth Report to Congress Effectiveness of Occupant Protection Systems and Their Use. Analyses of that type have shown, over the years, a consistent and statistically significant 12 percent overall fatality reduction for air bags. That said, I think the authors have selected the right statistical methods (including the use of SUDAAN for computing sampling errors) for their data and their analysis goals, and have clearly explained how their analyses work and how the results should be interpreted. Reference Fifth/Sixth Report to Congress Effectiveness of Occupant Protection Systems and Their Use, NHTSA Report No. DOT HS , Washington, DC, 2001, pp Reply to Discussion of Who Wants Airbags? The Crashworthiness Data System (CDS) is a well-conducted stratified random sample of highways accidents in the United States. All accidents from which there has been a towed vehicle and/or damage to persons or property have a chance of being in the dataset. The Fatality Analysis Reporting System (FARS) is another high-quality dataset that contains information for all accidents in which there was at least one death caused by the crash. Analysis conducted using CDS will indeed produce estimates with higher standard errors than analyses conducted with a larger dataset, but the logistic regression procedure accounts for the larger variation in its assessment of statistical significance. The only problem with using a smaller dataset is a possible lack of power; for instance, the inability to distinguish effects as statistically significant. However, the analyses reported using CDS, with more than 22,000 records for front-seat passengers ages 16 or older, demonstrate that the data are large enough to capture many significant effects; seatbelt use, impact speed, direction of collision, etc., are all seen to have the expected effect on the probability of death. Although the FARS dataset will give more precise results, they are not accurate, as they make conclusions about the wrong population. Do we want to reduce the probability of death overall, or merely the probability of death for occupants who are in collisions in which there is at least one fatality? If a front-seat occupant wishes to ask the question, If I get in an accident, am I less likely or more likely to die, if I have an airbag? the proper way to answer this question is with the CDS dataset. With the FARS dataset, the question one can answer is, If I get in a highway accident in which there is at least one fatality, am I less likely or more likely to die, if I have an airbag? It seems paradoxical that these two questions can have different answers, but they do. The CDS dataset can show us that in low-speed collisions, having an airbag increases the probability of death, and this is especially true for unseatbelted occupants where the main collision is from the side. This fact cannot be seen using the FARS dataset, because the information about low-speed crashes in which there was not a fatality is missing. The increase in risk to occupants in low-speed crashes, due to airbags, cannot be demonstrated. Therefore, the CDS dataset is the proper tool to assess risk of death in an accident. The distinction is not at all obvious at the outset, and no blame should be attached to those who chose to use FARS for the original analyses. The airbag risk analysis is a great example of the subtleties and challenges of quantitative reasoning, but it is clearly demonstrated that the CDS analyses better reflect the truth about airbags. Not only can we reproduce the results from the analyses with FARS, but there are convincing explanations for the disparate results. This country s commitment to airbags as a safety device needs to be reexamined immediately. 16 VOL. 18, NO. 2, 2005