Chapter 8: Measuring

Transcription

1 Chapter 8: Measuring Measurement Basics Thought Question Give five examples of properties that are easy to measure. height, weight, blood pressure, number of absences, number of car accidents Thought Question Give five examples of properties that are difficult to measure. happiness, amount of leisure time spent, student learning, intelligence, worker motivation Terminology Measurement: assigned number (numerical variable) that represents a property of a person or thing. Instrument: tool used to make a measurement. 1

2 Example: Weight Goal: measure the weight of a person. Instrument: scale Unit of measurement: pounds (kilograms) Variable: weight of the person in pounds (kilograms) Questions to Ask Exactly how is the variable defined? Is the variable a valid way to describe the property it claims to measure? How accurate are the measurements? Example: Unemployment Each month the Bureau of Labor Statistics announces the unemployment rate for the previous month, using this calculation: unemployment rate = number of people unemployed number of people in the labor force. What does it mean to be in the labor force? What does it mean to be unemployed? 2

3 Valid and Invalid Measurements Terminology Valid Measurement: relevant or appropriate representation of a property. Count: the number of occurrences. Rate: the fraction/proportion/percentage at which something occurs. Predictive Validity: the measurement can be used to predict success on tasks that are related to the property measured. Is it valid to measure weight with a scale reading in pounds? Yes Is it valid to measure musical ability by recording the number of siblings someone has? No Example: Sports Injuries In 2001, roughly 66,200,000 people participated in swimming, and 130,362 people were injured. Roughly 8,000,000 people participated in skateboarding, and 56,435 people were injured. Which sport had more injuries, swimming or skateboarding? swimming Should we conclude that swimming is more dangerous than skateboarding? Not necessarily, as many more people participated in swimming compared to skateboarding. So is the number of injuries a valid measure of the dangerousness of a sport? No A more valid way to measure dangerousness of a sport: measure injuries per 1000 participants. 3

4 Calculate the injury rate for both sports. swimming: 130,362/66, injuries per 1000 participants skateboarding: 56,435/8, injuries per 1000 participants What can you conclude about the dangerousness of the two sports? Swimming is less dangerous than skateboarding. Example: SAT Is the SAT a valid measure of readiness for college? Is this question easy to answer? No, readiness is a vague concept that probably combines inborn intelligence, learned knowledge, study and test-taking skills, and motivation to work at academic subjects. A simpler question: do SAT scores help predict students success in college? Why is this an easier question to answer? Success in college is a clear concept, measured by whether students graduate and by their college grades. Higher SAT scores = more likely to graduate and earn higher grades. So, SAT scores have predictive validity as measures of readiness for college. Important Questions How accurately do SAT scores predict college grades? For what groups do SAT scores have predictive validity? 4

5 Accurate and Inaccurate Measurements Terminology Bias: measurement systematically tends to overestimate or underestimate the true value. Random Error: repeated measurements on the same individual give different results. Reliable Measurement: random error is small. The measurement is repeatable. Example: Reading In 30 seconds, read the following passage and count the number of F s: THE NECESSITY OF TRAINING HANDS FOR FIRST-CLASS FARMS IN THE FATHERLY HANDLING OF FRIENDLY FARMS LIVESTOCK IS FOREMOST IN THE MINDS OF FARM OWNERS. SINCE THE FOREFATHERS OF THE FARM OWNERS TRAINED THE FARM HANDS FOR FIRST-CLASS FARMS IN THE FATHERLY HANDLING OF FARM LIVESTOCK, THE OWNERS OF THE FARMS FEEL THEY SHOULD CARRY ON WITH THE FAMILY TRADITION OF TRAINING FARM HANDS OF FIRST-CLASS FARMS IN THE FATHERLY HANDLING OF FARM LIVE- STOCK BECAUSE THEY BELIEVE IT IS THE BASIS OF GOOD FUNDAMENTAL FARM EQUIPMENT. Results: Type of error: random error Example: Scale Originally your bathroom scale always reads 3 pounds too high. Equation relating your true weight and your measured weight: measured weight = true weight + 3 pounds Type of error: bias 5

6 Now your bathroom scale also sometimes sticks. First measurement: 1 pound too low Equation relating your true weight and your measured weight: measured weight = true weight + 3 pounds - 1 pound Second measurement: 1 pound too high Equation relating your true weight and your measured weight: measured weight = true weight + 3 pounds + 1 pound Type of error: bias and random error General Relationship measured value = true value + bias + random error Example: Valid But Not Reliable Using a sticky scale to measure weight. Example: Reliable But Not Valid Measuring intelligence by skull volume. 6

7 Reducing Bias and Improving Reliability To reduce bias, improve your instrument (not always easy!). There is no such thing as a perfectly reliable measurement instrument - there will always be random error! To improve reliability, average several repeated measurements. 7

8 Chapter 8 Exercises 1. The National Highway Traffic Safety Administration reports that in 2006, between the hours of 6 AM and 6 PM, 8160 occupants of motor vehicles who were wearing a restraint died in motor vehicle accidents and 7064 who were not wearing a restraint died. These numbers suggest that not using a restraining device is safer than using one. The counts aren t fully convincing, however. What rates would you like to know to compare the safety of using a restraint with not using one? Compare death rates among occupants of motor vehicles involved in an accident who were wearing a restraining device with the rates for those who were not wearing a restraining device. 2. Congress wants the medical establishment to show that progress is being made in fighting cancer. Some variables that might be used are: 1. Total deaths from cancer. These have risen sharply over time, from 331,000 in 1970, to 505,000 in 1990, and to 553,888 in The percentage of all Americans who died from cancer. The percentage of deaths due to cancer rose steadily, from 17.2% in 1970 to 23.5% in 1990, then leveled off 23.1% in The percentage of cancer patients who survive for 5 years from the time the disease was discovered. These rates are rising slowly. For whites, the 5-year survival rate was 50.3% in the 1974 to 1976 period and 64.1% from 1995 to None of these variables is fully valid as a measure of the effectiveness of cancer treatment. Explain why Variables 1 and 2 could increase even if the treatment is getting more effective, and why Variable 3 could increase even if treatment is getting less effective. Variable 1: Because cancer is largely a disease of old age, cancer deaths increase as the population becomes older. Variable 2: Death rates from cancer rise as the health of the population improves and fewer people die of other causes. So progress against childhood epidemics and heart disease raise cancer death rates. Variable 3: Survival rates are more relevant to treatment of cancer. But survival time increases when the disease is discovered earlier, even if treatment is ineffective. Thus, this variable responds to factors other than the success of the treatment. 8