Lecture 15 - Curve Fitting Techniques Topics curve fitting motivtion liner regression Curve fitting - motivtion For root finding, we used given function to identify where it crossed zero where does fx ( ) 0?? Q: Where does this given function f( x) come from in the first plce? Anlyticl models of phenomen (e.g. equtions from physics) Crete n eqution from oserved dt 1) Interpoltion (connect the dt-dots) If dt is relile, we cn plot it nd connect the dots This is piece-wise, liner interpoltion This hs limited use s generl function f( x) Since its relly group of smll f( x) s, connecting one point to the next nd, it doesn t work very well for dt tht hs uilt in rndom error ) Curve fitting - cpturing the trend in the dt Let s drw line through the dt f(x) x + Interpoltion Curve Fitting A stright line is descried genericlly y f(x) x + The gol is to identify the coefficients nd such tht f(x) fits the dt Lecture 15 - Curve Fitting Techniques pge 13 of 19
other exmples height of dropped oject Oxygen in soil pore pressure time Profit temperture soil depth pid lor hours Is stright line suitle for ech of these cses? No. But we re not stuck with just stright line fits. Tht is just where we ll strt. Liner curve fitting (liner regression) Given the generl form of stright line????? f( x) x + How cn we pick the coefficients tht est fits the line to the dt? First question: Wht mkes prticulr stright line good fit? Why does the lue line pper to us to fit the trend etter? Consider the distnce etween the dt nd points on the line Add up the length of ll the red nd lue verticle lines This is n expression of the error etween line nd dt The one line tht provides minimum error is then the est stright line Lecture 15 - Curve Fitting Techniques pge 14 of 19
Quntifying error in curve fit ssumptions: 1) positive or negtive error hve the sme error vlue (dt point is ove or elow the line) ) Weigh greter errors more hevily we cn do oth of these things y squring the distnce denote dt vlues s (x, y) >> denote points on the line s (x, f(x)) sum the error t the four dt points (x,y ) (x 4,y 4 ) (x,f(x )) (x 4,f(x 4 )) err ( d i ) ( y 1 fx ( 1 )) + ( y fx ( )) + ( y 3 fx ( 3 )) + ( y 4 fx ( 4 )) now sustitute f( x) x + # dt points # dt points err ( y i fx ( i )) ( y i ( x i + ) ) i 1 i 1 find the est line minimize the error etween line nd dt points This is clled the lest squres pproch, since we minimize the squre of the error. # dt points n minimize err ( y i ( x i + ) )???????????????????? i 1 time to pull out the clculus... finding the minimum of function 1) derivtive descries the slope ) slope zero is minimum > tke the derivtive with respect to nd, set ech to zero Lecture 15 - Curve Fitting Techniques pge 15 of 19
Find nd so tht these two equtions oth 0 re-write these two equtions x i + x i put these into mtrix form n err ---------- x i ( y i x i ) 0 i 1 n err ---------- y ( i x i ) 0 i 1 ( x i y i ) x + *n y i i????????????? n x i x i x i y i ( x i y ) i wht s unknown? we hve the dt points ( x i, y i ) for i 1,..., n so unknows re nd Good news, we lredy know how to solve this prolem rememer Gussin elimintion?? A n x i, X, B x i x i so AX B y i ( x i y ) i using uilt in MATLAB mtrix inversion, the coefficients nd re solved >> X inv(a)*b?????? Note: A, B, nd X re not the sme s,, nd x Let s test this with n exmple: Lecture 15 - Curve Fitting Techniques pge 16 of 19
Here s some dt i 1 3 4 5 6 x y 0 0.5 1.0 1.5.0.5 0 1.5 3.0 4.5 6.0 7.5 n 6 x i 7.5, y i.5 x i 13.75, x i y i 41.5 Plugging into the mtrix form gives us: 6 7.5 7.5 13.75.5 Note: we re using x, NOT 41.5 i ( x i ) inv 6 7.5 7.5 13.75 *.5 41.5 or use Gussin elimintion... The solution is 0 > f( x) 3x + 0 3 This fits the dt exctly. Tht is, the error is zero. Usully this is not the outcome. Usully we hve dt tht does not exctly fit stright line. here s n exmple with some noisy dt x [0.5 1 1.5.5] y [-0.436-0.1656 3.153 4.7877 4.8535 8.6909] 6 7.5 7.5 13.75 0.8593 41.6584 inv 6 7.5 7.5 13.75 * 0.8593 41.6584 Lecture 15 - Curve Fitting Techniques pge 17 of 19
0.975 3.561 so our fitted curve is f( x) 4.15 x.05 Here s plot of the dt nd the curve fit: Lecture 15 - Curve Fitting Techniques pge 18 of 19
So...wht do we do when stright line is not suitle model? Profit pid lor hours Stright line will not predict diminishing returns tht dt shows We strted the liner curve fit y choosing generic form of the stright line f(x) x + This is just one kind of function. There re n infinite numer of generic forms we could choose from for lmost ny shpe we wnt. Let s strt with simple extension to the liner regression concept Lecture 15 - Curve Fitting Techniques pge 19 of 19