Linear Regression The Method of Least Squares Data Fitting

Transcription

1 Lnear Regresson The Method of Least Squares Data Fttng by Dr. James E. Parks Department of Physcs and Astronomy 401 Nelsen Physcs Buldng The Unversty of Tennessee Knoxvlle, Tennessee Copyrght August, 000 by James Edgar Parks* *All rghts are reserved. No part of ths publcaton may be reproduced or transmtted n any form or by any means, electronc or mechancal, ncludng photocopy, recordng, or any nformaton storage or retreval system, wthout permsson n wrtng from the author. Objectve: The objectve of ths experment s to understand the method of lnear regresson or the method of least squares. Theory Often measurements are made of a quantty Y that depends on another sngle ndependent quantty X that s also measured. Many tmes the relatonshp between the two measured quanttes s a lnear relatonshp or at least t s to a frst approxmaton. The mathematcal equaton that descrbes the relatonshp between the dependent varable Y and the ndependent varable X s sad to be a lnear equaton or that Y s a lnear functon of X. If the quantty Y s drectly proportonal to the quantty X, then the relatonshp s a lnear relatonshp, and when Y s plotted on a graph as a functon of X, the graph wll result n a straght lne. Such a lnear relatonshp s gven by the equaton Y= A + B X. (1) Equaton (1) s recognzed as the equaton of a straght lne where A s the y-ntercept value and B s the slope of the lne. When a number, N, of ndvdual pars of data, X and Y, are measured over a range of values and the relatonshp s lnear, then the parameters A and B n the equaton that descrbes the relatonshp can be found by a mathematcal procedure referred to as lnear regresson or the method of least squares fttng. Ths procedure s convenently preformed automatcally by spreadsheet programs, however, n ths experment the method tself wll be studed and the procedure wll be

2 performed manually. The spreadsheet wll be used to make the smple calculatons on multple data entres and to plot the data for vewng. The method of least squares can be llustrated wth the followng consderatons. Suppose a set of n calbraton measurements are made n whch the th measurement determnes Y for an nput value of X. The values of A and B are then to be determned n such a way that the sum of the squares of the dfferences between the measured values Y and the calculated values Y(X ) for each of the X values s a mnmum. The measurements result n a table of values represented by X 1 Y 1 X Y X 3 Y 3 X 4 Y 4 X Y X n Y n For each ndependent value, X, the equaton for the straght lne n terms of the unknown parameters, A and B, would produce a value equal Y(X ) = A + B X gvng rse to a new column of values, although unknown at ths tme, thus producng X 1 Y 1 A + B X 1 X Y A + B X X 3 Y 3 A + B X 3 X 4 Y 4 A + B X 4 X Y A + B X X n Y n A + B X n The method of least squares then looks at the squares of the dfferences between the second and thrd columns (to elmnate negatve numbers or to make the dfference postve n all cases). Ths can gve rse to a fourth column of values

3 X 1 Y 1 A + B X 1 [Y 1 -(A + B X 1 )] X Y A + B X [Y -(A + B X )] X 3 Y 3 A + B X 3 [Y 3 -(A + B X 3 )] X 4 Y 4 A + B X 4 [Y 4 -(A + B X 4 )] X Y A + B X [Y -(A + B X )] X n Y n A + B X n [Y n -(A + B X n )] The method of least squares then seeks to mnmze the sum of ths fourth column of numbers by the proper choce of values for A and B. Ths sum, S, can be wrtten =n S= [Y -(A + B X )] () =1 Usng calculus based procedures to fnd mnma and maxma, the values of A and B to make the sum S a mnmum, are found by dfferentatng the sum, S, wth respect to A and B respectvely yeldng the followng two equatons: =n = -[Y -(A + B X )] da =1 (3) =n = -X [Y -(A + B X )]. db (4) =1 To fnd the mnmum, these equatons are set equal to zero to yeld =n da = -[Y - A - B X ] = 0 (5) db = =n =1 -[XY -AX - B X] = 0. (6) =1 These equatons may then be smplfed and rewrtten as =n =n Y -n A - B X = 0 (7) =n =1 =n =1 XY -A X - B X = 0. (8) =1 =1 =n =1

4 These equatons can then be rearranged wth the unknown parameters A and B on the left to yeld =n n A+[ X ] B= Y (9) =1 =n =1 =n =n =n =1 =1 =1 [ X ] A+ [ X ] B= XY. (10) In ths more famlar form, the equatons can then be solved for the unknowns A and B. From a practcal standpont n order to solve for A and B, the followng data table s more useful. X 1 Y 1 (X 1 ) X 1 Y 1 X Y (X ) X Y X 3 Y 3 (X 3 ) X 3 Y 3 X 4 Y 4 (X 4 ) X 4 Y 4 X Y (X ) X Y X n Y n (X n ) X n Y n ΣX ΣY Σ(X ) ΣX Y The last row s the sum of each of the columns and contans the parameters needed to solve equatons (9) and (10) for A and B. A and B are then gven by the equatons and Y X - X XY A = n X - X X n XY - X Y B = n X - X X (11). (1) Procedure 1. Open up an Excel spreadsheet and enter n Columns A and B the values shown for X and Y as shown n Fgure 1.. Use the Chart optons to make a graph of Y versus X as shown n Fgure.

5 A B C D 1 X Y X XY Fgure 1. Excel spreadsheet wth values of Y versus X Y Value X Value Fgure 1. Excel graph of Y versus X. 3. In columns C and D, compute the values of X and XY. 4. In cells A13, B13, C13, and D13, fnd the sums of the values for X, Y, X, and X Y. 5. Use the values for n, ΣX, ΣY, ΣX, and ΣX Y n equatons (11) and (1) to fnd A and B. 6. On your Excel plot, rght clck your cursor on one of the data ponts and choose the Add Trendlne opton. Choose the Lnear Type and the Optons Tab. In the Optons wndow clck on the Dsplay equaton on chart button and then OK.

6 7. Compare the values you found for A and B wth those values Excel found automatcally wth the Trendlne functon. How do they compare?