PHYS2150 Lecture 3. Comparing values Rejecting data. Weighted averages

Transcription

1 PHYS2150 Lecture 3 Comparing values Rejecting data Weighted averages

2 COMPARING WITH KNOWN VALUE: Measure: g = [997 ± 32] cm/s 2 = x± δx Usually negligible Known value: cm/s 2 = x0± δx0 Discrepancy is (x-x0)± δ(x-x0), where [δ(x-x0)] 2 = (δx) 2 + (δx0) 2 Significance of Discrepancy (in sigmas) Use erf table to determine agreement confidence level: 62% agreement: good! If the significance is less than 1.0 (measurement was within one standard deviation of the correct one), then the measured value is near enough to the known value to be acceptable.

3 Comparing with Known Values Percent Error = Measured Accepted x 100% Accepted This is more of a ball park estimate. In general, this quantity should be less than 10% in this lab. This gives you another way to look at how far apart the two quantities are.

4 ANOTHER EXAMPLE: e/m

5 ANOTHER EXAMPLE: e/m artifact

6 ANOTHER EXAMPLE: e/m

7 ANOTHER EXAMPLE: e/m Note: Systematic error comes out to be which is much closer to the std. dev. than the uncertainty on the mean. For only 14 data points, std. dev. should be used, not uncertainty on the mean.

8 ANOTHER EXAMPLE: e/m

9 ANOTHER EXAMPLE: e/m

10 ANOTHER EXAMPLE: e/m Compare measured value [1.946 ± (stat.) ± (sys.)]x10-11 C/kg to accepted value x C/kg Discrepancy from the accepted value is 1.946x x C/kg = 0.187x10-11 C/kg Significance of discrepancy = Discrepancy= C/kg σ total 2 ( 0.019) + (0.064) 2 C/kg = 2.8 (often stated as off by 2.8 sigma ) If uncertainties calculated correctly, should get within 1 σ 68% of time. Otherwise, something else is wrong.

11 ANOTHER EXAMPLE: e/m What is the probability of value being further than 2.8σ from the mean for a gaussian distribution? Probability of being inside 2.8σ is P(2.8σ) = 99.49% from table p.287 Taylor. (Check this in the table!) Probability of being outside 2.8σ is ( ) = 0.51% It is very unlikely that a reasonably measured quantity would be that far from the mean. Either a mistake or underestimated systematic uncertainties. The discrepancy between the measured and accepted value was significant. (bad) Note: If they had used the std. dev instead of the uncert. on mean, the significance would be 1.97σ. Probably more correct. (still bad) Note: If this happens to you, you will need to talk more in your report about what sources of error might have caused this. Ask questions if you are not sure.

12 REJECTING DATA BE VERY CAREFUL HERE! You are treading in the footsteps of a long line of practitioners of pathological science. Generally, you should have external reasons for cutting data In reality, even that s not enough: do you only go searching for problems when you get a result you didn t expect? Your analysis is biased! All prescriptions for deciding on the validity of data points are controversial! I ll describe one that s common in textbooks (but not in real life): Chauvenet s criterion. First, a cautionary tale.

13 HOW TO LOOK FOR A PARTICLE Those of you doing the K mesonexperiment have already seen this Look in high-energy collisions for events with multiple output particles that could be decay products (displaced from primary interaction, if particle is long-lived as with the K 0 ) Reconstruct a relativistic invariant mass from the momenta of the decay products Make a histogram of the masses from candidate events Look for a peak, indicating a state of well-defined mass

14 ONE PEAK OR TWO? CERN experiment in late 1960s observed A 2 mesons Particle appeared to be a doublet Statistical significance of split is very high There is really only one particle.

15 HOW DID IT HAPPEN? In an early run, a dip showed up. It was a statistical fluctuation, but people noticed it and suspected it might be real. Subsequent runs were looked at as they came in. If no dip showed up, the run was investigated for problems. There s usually a minor problem somewhere in a complicated experiment, so most of these runs were cut from the sample. When a dip appeared, they didn t look as carefully for a problem. So an insignificant fluctuation was boosted into a completely wrong discovery. Lesson: Don t let result influence which data sets you use.

16 CHAUVENET S CRITERION Assume data (xi; i=1,n) come from a gaussian distribution. Determine mean <x> and standard deviation σ. Assuming gaussian distribution, calculate how many data points (fraction times N) should be outside a particular number of standard deviations. Pick a threshold Chauvenet says 0.5 In real life you would probably want to use a smaller number) reject data points farther from the mean So if you use 0.5 as your threshold, you would throw out data points beyond a number of sigma where you expect only 0.5 data points.

17 CHAUVENET S CRITERION Example: Have 10 data points. Know that 5% of data points should be outside 2 σ, 0.5 data points. So, throw out an event if it s more than 2σ from the mean. Can t do this iteratively (recaculate mean and sigma with remaining data, then throw out some more points)! It would make your measurement meaningless.

18 A FINAL COMMENT ON REJECTING DATA Rejecting data is a last resort. If you are suspicious of a data point, you should first: If you can, take more data! Ask what could have gone wrong (but be careful that you aren t going on a witch hunt) Is there a problem with the way you recorded/calibrated that could cause this data point to be different? Whatever you decide, lay it all out in the report, including the result including the rejected data. Be honest about it. NEVER reject data points because you don t like the answer!

19 WEIGHTED AVERAGES If you measure the same thing twice, and the errors are different, how do you combine the results? A proper averaging gives more weight to measurements with smaller uncertainties (they are more right ). The reported error on the average must reflect this in some way. Weighted averages are the most basic form of fitting.

20 WEIGHTED AVERAGES We showed that the mean of measurements gives the best measurement of the true value This assumed that each measurement had the same uncertainty! How to combine when they are different? Assume two measurements of g: g1=(980.2±3.7) cm/s 2 g2=(977.4±1.9) cm/s 2 If unweighted, mean = cm/s 2, but this ignores the fact that measurement 2 has smaller uncertainty so the true value is likely to be closer to it than to measurement 1.

21 TURNING MAXIMUM LIKELIHOOD ARGUMENT INTO A WEIGHTED AVERAGE Weights go as 1/σ 2, so more precise measurements count much more than less precise ones.

22 USING WEIGHTED AVERAGES Before averaging, it s a good idea to check compatibility of the results: Same idea as discrepancy in Lecture 2. Two measured values: (x A ± σ A ) and (x B ± σ B ) Difference is x A x B Significance of difference (in σ s) is δ = x A x B σ 2 A + σ 2 B σ Can convert significance of difference in σ to probability using erf (calculated or table)

23 ARE OUR MEASUREMENTS CONSISTENT? The values are within their combined error bars.

24 TAKE THE AVERAGE: Straight average is m/s2 Weighted average 9.780±0.017 Weighted average is much closer to the more precise measurement Uncertainty is lower than on the individual measurements: we have more information in the weighted average.

25 WEIGHTED AVERAGE: WHEN? NO, if: You don t know the intrinsic uncertainty, or The intrinsic uncertainty is the same for all measurements You need to calculate the standard deviation to determine the uncertainty YES, if: You know the intrinsic uncertainties, AND They are not the same for all measurements