# Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork

Save this PDF as:

Size: px
Start display at page:

Download "Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork"

## Transcription

1 Solutios to Selected Problems I: Patter Classificatio by Duda, Hart, Stork Joh L. Weatherwax February 4, 008 Problem Solutios Chapter Bayesia Decisio Theory Problem radomized rules Part a: Let Rx be the average risk give measuremet x. The sice this risk depeds o the actio ad correspodigly the probability of takig such actio, this risk ca be expressed as m Rx Ra i xpa i x The the total risk over all possible x is give by the average of Rx or R Ω x i m Ra i xpa i xpxdx i Part b: Oe would pick Pa i x { i ArgMii Ra i x 0 i aythig else Thus for every x pick the actio that miimizes the local risk. This amouts to a determiistic rule i.e. there is o beefit to usig radomess.

2 Part c: Yes, sice i a determiistic decisio rule settig radomizig some decisios might further improve performace. But how this would be doe would be difficult to say. Problem 3 Let 0 i j λα i ω j λ r i c + λ s otherwise The sice the defiitio of risk is expected loss we have Rα i x c λa i ω j Pω j x i,,..., c i Rα c+ x λ r Now for i,,..., c the risk ca be simplified as Rα i x λ s c j, j i λa i ω j Pω j x λ s Pω i x Our decisio fuctio is to choose the actio with the smallest risk, this meas that we pick the reject optio if ad oly if This is equivalet to λ r < λ s Pω i x i,,..., c Pω i x < λ r λ s i,,..., c This ca be iterpreted as if all posterior probabilities are below a threshold λr λ s we should reject. Otherwise we pick actio i such that This is equivalet to λ s Pω i x < λ s Pω i x j,,..., c, j i Pω i x > Pω j x j i This ca be iterpreted as selectig the class with the largest posteriori probability. I summary we should reject if ad oly if Pω i x < λ r λ s i,,..., c 3 Note that if λ r 0 there is o loss for rejectig equivalet beefit/reward for correctly classifyig ad we always reject. If λ r > λ s, the loss from rejectig is greater tha the loss from missclassifig ad we should ever reject. This ca be see from the above sice λr λ s > so λr λ s < 0 so equatio 3 will ever be satisfied.

3 Problem 3 the probability of error Part a: We ca assume with out loss of geerality that µ < µ. If this is ot the case iitially chage the labels of class oe ad class two so that it is. We begi by recallig the probability of error P e P e PĤ H H PH + PĤ H H PH ξ px H dx PH + px H dx PH. Here ξ is the decisio boudary betwee the two classes i.e. we classify as class H whe x < ξ ad classify x as class H whe x > ξ. Here PH i are the priors for each of the two classes. We begi by fidig the Bayes decisio boudary ξ uder the miimum error criterio. From the discussio i the book, the decisio boudary is give by a likelihood ratio test. That is we decide class H if px H px H PH PH ξ C C, C C ad classify the poit as a member of H otherwise. The decisio boudary ξ is the poit at which this iequality is a equality. If we use a miimum probability of error criterio the costs C ij are give by C ij δ ij, ad the decisio boudary reduces whe PH PH to pξ H pξ H. If each coditioal distributios is Gaussia px H i Nµ i, σi, the the above expressio becomes πσ exp{ ξ µ } σ πσ exp{ ξ µ }. If we assume for simplicity that σ σ σ the above expressio simplifies to ξ µ ξ µ, or o takig the square root of both sides of this we get ξ µ ±ξ µ. If we take the plus sig i this equatio we get the cotradictio that µ µ. Takig the mius sig ad solvig for ξ we fid our decisio threshold give by ξ µ + µ. Agai sice we have uiform priors PH PH, we have a error probability give by or P e ξ πσp e exp{ x µ }dx + πσ σ ξ ξ exp{ x µ }dx + σ σ πσ exp{ ξ x µ }dx. σ exp{ x µ }dx. σ To evaluate the first itegral use the substitutio v x µ σ so dv dx σ, ad i the secod use v x µ σ so dv dx σ. This the gives πσp e σ µ µ σ e v dv + σ e v dv. µ µ σ

4 or P e µ µ σ e v dv + π e v dv. π µ µ σ Now sice e v is a eve fuctio, we ca make the substitutio u v ad trasform the secod itegral ito a itegral that is the same as the first. Doig so we fid that P e µ µ σ π e v dv. Doig the same trasformatio we ca covert the first itegral ito the secod ad we have the combied represetatio for P e of P e µ µ σ π e v dv π µ µ σ e v dv. Sice we have that µ < µ, we see that the lower limit o the secod itegral is positive. I the case whe µ > µ we would switch the ames of the two classes ad still arrive at the above. I the geeral case our probability of error is give by P e π a e v dv, with a defied as a µ µ. To match exactly the book let v u σ so that dv du ad the above itegral becomes P e e u / du. π I terms of the error fuctio which is defied as erfx x π dt, we have that 0 e t [ ] [ ] P e [ π a e v dv a π 0 ] e v dv erf µ µ σ. a π 0 e v dv a π 0 e v dv Sice the error fuctio is programmed ito may mathematical libraries this form is easily evaluated. Part b: From the give iequality P e π a e t / dt πa e a /, ad the defiitio a as the lower limit of the itegratio i Part a above we have that P e π µ µ σ e µ µ 4σ σ π µ µ exp{ 8 µ µ σ } 0,

5 as µ µ +. This ca happe if µ σ ad µ get far apart of σ shriks to zero. I each case the classificatio problem gets easier. Problem 3 probability of error i higher dimesios As i Problem 3 the classificatio decisio boudary are ow the poits x that satisfy x µ x µ. Expadig each side of this expressio we fid x x t µ + µ x x t µ + µ. Cacelig the commo x from each side ad groupig the terms liear i x we have that x t µ µ µ µ. Which is the equatio for a hyperplae. Problem 34 error bouds o o-gaussia data Part a: Sice the Bhattacharyya boud is a specializatio of the Cheroff boud, we begi by cosiderig the Cheroff boud. Specifically, usig the boud mia, b a β b β, for a > 0, b > 0, ad 0 β, oe ca show that Perror Pω β Pω β e kβ for 0 β. Here the expressio kβ is give by β β kβ µ µ t [βσ + βσ ] µ µ + l βσ + βσ. Σ β Σ β Whe we desire to compute the Bhattacharyya boud we assig β /. For a two class scalar problem with symmetric meas ad equal variaces that is where µ µ, µ +µ, ad σ σ σ the expressio for kβ above becomes kβ β β µ [ βσ + βσ ] µ + βσ l + βσ σ β σ β β βµ. σ Sice the above expressio is valid for all β i the rage 0 β, we ca obtai the tightest possible boud by computig the maximum of the expressio kβ sice the domiat β expressio is e kβ. To evaluate this maximum of kβ we take the derivative with respect to β ad set that result equal to zero obtaiig d β βµ 0 dβ σ

6 which gives β 0 or β /. Thus i this case the Cheroff boud equals the Bhattacharra boud. Whe β / we have for the followig boud o our our error probability Perror Pω Pω e µ σ. To further simplify thigs if we assume that our variace is such that σ µ, ad that our priors over class are take to be uiform Pω Pω / the above becomes Perror e /

7 Chapter 3 Maximum Likelihood ad Bayesia Estimatio Problem 34 I the problem statemet we are told that to operate o a data of size, requires f umeric calculatios. Assumig that each calculatio requires 0 9 secods of time to compute, we ca process a data structure of size i 0 9 f secods. If we wat our results fiished by a time T the the largest data structure that we ca process must satisfy 0 9 f T or f 0 9 T. Covertig the rages of times ito secods we fid that T { sec, hour, day, year} { sec, 3600 sec, sec, sec}. Usig a very aive algorithm we ca compute the largest such that f by simply icremetig repeatedly. This is doe i the Matlab script prob 34 chap 3.m ad the results are displayed i Table XXX. Problem 35 recursive v.s. direct calculatio of meas ad covariaces Part a: To compute ˆµ we have to add, d-dimesioal vectors resultig i d computatios. We the have to divide each compoet by the scalar, resultig i aother computatios. Thus i total we have calculatios. d + + d, To compute the sample covariace matrix, C, we have to first compute the differeces x k ˆµ requirig d subtractio for each vector x k. Sice we must do this for each of the vectors x k we have a total of d calculatios required to compute all of the x k ˆµ differece vectors. We the have to compute the outer products x k ˆµ x k ˆµ, which require d calculatios to produce each i a aive implemetatio. Sice we have such outer products to compute computig them all requires d operatios. We ext, have to add all of these outer products together, resultig i aother set of d operatios. Fially dividig by the scalar requires aother d operatios. Thus i total we have calculatios. d + d + d + d d + d, Part b: We ca derive recursive formulas for ˆµ ad C i the followig way. First for ˆµ we ote that + ˆµ + x k x x k + k k + x + + ˆµ. +

8 Writig + as expected. as the above becomes ˆµ + + x + + ˆµ + ˆµ ˆµ + x + ˆµ, + To derive the recursive update formula for the covariace matrix C we begi by recallig the defiitio of C of C + + x k ˆµ + x k ˆµ +. k Now itroducig the recursive defiitio of the mea ˆµ + as computed above we have that C + + k + x k ˆµ + x + ˆµ x k ˆµ + x + ˆµ x k ˆµ x k ˆµ k + + k Sice ˆµ k x k we see that + + x + ˆµ x k ˆµ ˆµ x k 0 or k k x k ˆµ x + ˆµ + x + + ˆµ x + ˆµ. k ˆµ x k 0, Usig this fact the secod ad third terms the above ca be greatly simplified. For example we have that + x k ˆµ x k ˆµ + x + ˆµ x + ˆµ k k Usig this idetity ad the fact that the fourth term has a summad that does ot deped o k we have C + give by C + x k ˆµ x k ˆµ + x + ˆµ x + ˆµ k + x + ˆµ x + ˆµ + C C C + k + x + ˆµ x + ˆµ x + ˆµ x + ˆµ x + ˆµ x + ˆµ + C + + x + ˆµ x + ˆµ.

9 which is the result i the book. Part c: To compute ˆµ usig the recurrece formulatio we have d subtractios to compute x ˆµ ad the scalar divisio requires aother d operatios. Fially, additio by the ˆµ requires aother d operatios givig i total 3d operatios. To be compared with the d + operatios i the o-recursive case. To compute the umber of operatios required to compute C recursively we have d computatios to compute the differece x ˆµ ad the d operatios to compute the outer product x ˆµ x ˆµ, aother d operatios to multiply C by ad fially d operatios to add this product to the other oe. Thus i total we have d + d + d + d + d 4d + d. To be compare with d + d i the o recursive case. For large the o-recursive calculatios are much more computatioally itesive.

10 Chapter 6 Multilayer Neural Networks Problem 39 derivatives of matrix ier products Part a: We preset a slightly differet method to prove this here. We desire the gradiet or derivative of the fuctio φ x T Kx. The i-th compoet of this derivative is give by φ x i x i x T Kx e T i Kx + xt Ke i where e i is the i-th elemetary basis fuctio for R, i.e. it has a i the i-th positio ad zeros everywhere else. Now sice the above becomes e T i KxT x T K T e T i e T i Kx, φ x i e T i Kx + et i KT x e T i K + K T x. Sice multiplyig by e T i o the left selects the i-th row from the expressio to its right we see that the full gradiet expressio is give by φ K + K T x, as requested i the text. Note that this expressio ca also be proved easily by writig each term i compoet otatio as suggested i the text. Part b: If K is symmetric the sice K T K the above expressio simplifies to as claimed i the book. φ Kx, Chapter 9 Algorithm-Idepedet Machie Learig Problem 9 sums of biomial coefficiets Part a: We first recall the biomial theorem which is x + y x k y k. k If we let x ad y the x + y ad the sum above becomes, k k0 k0

11 which is the stated idetity. As a aside, ote also that if we let x ad y the x + y 0 ad we obtai aother iterestig idetity ivolvig the biomial coefficiets 0 k0 k k. Problem 3 the average of the leave oe out meas We defie the leave oe out mea µ i as µ i x j. This ca obviously be computed for every i. Now we wat to cosider the mea of the leave oe out meas. To do so first otice that µ i ca be writte as µ i j i x j j i x j x i j j ˆµ x i. x j x i With this we see that the mea of all of the leave oe out meas is give by as expected. µ i i i ˆµ ˆµ x i ˆµ ˆµ ˆµ, i x i Problem 40 maximum likelihood estimatio with a biomial radom variable Sice k has a biomial distributio with parameters, p we have that Pk p k p k. k

12 The the p that maximizes this expressio is give by takig the derivative of the above with respect to p settig the resultig expressio equal to zero ad solvig for p. We fid that this derivative is give by d dp Pk k kp k p k + k p k p k k. Which whe set equal to zero ad solve for p we fid that p k, or the empirical coutig estimate of the probability of success as we were asked to show.

### Properties of MLE: consistency, asymptotic normality. Fisher information.

Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

### Chapter 7 Methods of Finding Estimators

Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

### A probabilistic proof of a binomial identity

A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

### THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

### Soving Recurrence Relations

Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

### Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

### Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

### Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean

1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.

### Lesson 15 ANOVA (analysis of variance)

Outlie Variability -betwee group variability -withi group variability -total variability -F-ratio Computatio -sums of squares (betwee/withi/total -degrees of freedom (betwee/withi/total -mea square (betwee/withi

### Basic Elements of Arithmetic Sequences and Series

MA40S PRE-CALCULUS UNIT G GEOMETRIC SEQUENCES CLASS NOTES (COMPLETED NO NEED TO COPY NOTES FROM OVERHEAD) Basic Elemets of Arithmetic Sequeces ad Series Objective: To establish basic elemets of arithmetic

### SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

### I. Chi-squared Distributions

1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

### Overview of some probability distributions.

Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

### University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

### THE ARITHMETIC OF INTEGERS. - multiplication, exponentiation, division, addition, and subtraction

THE ARITHMETIC OF INTEGERS - multiplicatio, expoetiatio, divisio, additio, ad subtractio What to do ad what ot to do. THE INTEGERS Recall that a iteger is oe of the whole umbers, which may be either positive,

### Sequences and Series

CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

### Normal Distribution.

Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

### 0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

### 1. MATHEMATICAL INDUCTION

1. MATHEMATICAL INDUCTION EXAMPLE 1: Prove that for ay iteger 1. Proof: 1 + 2 + 3 +... + ( + 1 2 (1.1 STEP 1: For 1 (1.1 is true, sice 1 1(1 + 1. 2 STEP 2: Suppose (1.1 is true for some k 1, that is 1

### Maximum Likelihood Estimators.

Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

### Hypothesis testing. Null and alternative hypotheses

Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

### Output Analysis (2, Chapters 10 &11 Law)

B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

### SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

### In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

### UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

### Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

### Incremental calculation of weighted mean and variance

Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

### 1 Computing the Standard Deviation of Sample Means

Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

### Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

### Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is

0_0605.qxd /5/05 0:45 AM Page 470 470 Chapter 6 Additioal Topics i Trigoometry 6.5 Trigoometric Form of a Complex Number What you should lear Plot complex umbers i the complex plae ad fid absolute values

### Lesson 17 Pearson s Correlation Coefficient

Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

### CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations

CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad

### CS103X: Discrete Structures Homework 4 Solutions

CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible six-figure salaries i whole dollar amouts are there that cotai at least

### Modified Line Search Method for Global Optimization

Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

### , a Wishart distribution with n -1 degrees of freedom and scale matrix.

UMEÅ UNIVERSITET Matematisk-statistiska istitutioe Multivariat dataaalys D MSTD79 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multivariat dataaalys D, 5 poäg.. Assume that

### where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return

EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The

### Lecture 4: Cheeger s Inequality

Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

### Convexity, Inequalities, and Norms

Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

### BINOMIAL EXPANSIONS 12.5. In this section. Some Examples. Obtaining the Coefficients

652 (12-26) Chapter 12 Sequeces ad Series 12.5 BINOMIAL EXPANSIONS I this sectio Some Examples Otaiig the Coefficiets The Biomial Theorem I Chapter 5 you leared how to square a iomial. I this sectio you

### CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 8

CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 8 GENE H GOLUB 1 Positive Defiite Matrices A matrix A is positive defiite if x Ax > 0 for all ozero x A positive defiite matrix has real ad positive

### Mathematical goals. Starting points. Materials required. Time needed

Level A1 of challege: C A1 Mathematical goals Startig poits Materials required Time eeded Iterpretig algebraic expressios To help learers to: traslate betwee words, symbols, tables, ad area represetatios

### Factoring x n 1: cyclotomic and Aurifeuillian polynomials Paul Garrett <garrett@math.umn.edu>

(March 16, 004) Factorig x 1: cyclotomic ad Aurifeuillia polyomials Paul Garrett Polyomials of the form x 1, x 3 1, x 4 1 have at least oe systematic factorizatio x 1 = (x 1)(x 1

### Math C067 Sampling Distributions

Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

### CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

### Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

### FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. 1. Powers of a matrix

FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. Powers of a matrix We begi with a propositio which illustrates the usefuless of the diagoalizatio. Recall that a square matrix A is diogaalizable if

### GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.

GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all

### A Recursive Formula for Moments of a Binomial Distribution

A Recursive Formula for Momets of a Biomial Distributio Árpád Béyi beyi@mathumassedu, Uiversity of Massachusetts, Amherst, MA 01003 ad Saverio M Maago smmaago@psavymil Naval Postgraduate School, Moterey,

### Review: Classification Outline

Data Miig CS 341, Sprig 2007 Decisio Trees Neural etworks Review: Lecture 6: Classificatio issues, regressio, bayesia classificatio Pretice Hall 2 Data Miig Core Techiques Classificatio Clusterig Associatio

### PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

### 5 Boolean Decision Trees (February 11)

5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

### 2-3 The Remainder and Factor Theorems

- The Remaider ad Factor Theorems Factor each polyomial completely usig the give factor ad log divisio 1 x + x x 60; x + So, x + x x 60 = (x + )(x x 15) Factorig the quadratic expressio yields x + x x

### S. Tanny MAT 344 Spring 1999. be the minimum number of moves required.

S. Tay MAT 344 Sprig 999 Recurrece Relatios Tower of Haoi Let T be the miimum umber of moves required. T 0 = 0, T = 7 Iitial Coditios * T = T + \$ T is a sequece (f. o itegers). Solve for T? * is a recurrece,

### Asymptotic Growth of Functions

CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

### 5.3. Generalized Permutations and Combinations

53 GENERALIZED PERMUTATIONS AND COMBINATIONS 73 53 Geeralized Permutatios ad Combiatios 53 Permutatios with Repeated Elemets Assume that we have a alphabet with letters ad we wat to write all possible

### .04. This means \$1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

### Solving Logarithms and Exponential Equations

Solvig Logarithms ad Epoetial Equatios Logarithmic Equatios There are two major ideas required whe solvig Logarithmic Equatios. The first is the Defiitio of a Logarithm. You may recall from a earlier topic:

### Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

### Section 8.3 : De Moivre s Theorem and Applications

The Sectio 8 : De Moivre s Theorem ad Applicatios Let z 1 ad z be complex umbers, where z 1 = r 1, z = r, arg(z 1 ) = θ 1, arg(z ) = θ z 1 = r 1 (cos θ 1 + i si θ 1 ) z = r (cos θ + i si θ ) ad z 1 z =

### Descriptive Statistics

Descriptive Statistics We leared to describe data sets graphically. We ca also describe a data set umerically. Measures of Locatio Defiitio The sample mea is the arithmetic average of values. We deote

### Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find

1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.

### Repeating Decimals are decimal numbers that have number(s) after the decimal point that repeat in a pattern.

5.5 Fractios ad Decimals Steps for Chagig a Fractio to a Decimal. Simplify the fractio, if possible. 2. Divide the umerator by the deomiator. d d Repeatig Decimals Repeatig Decimals are decimal umbers

### 3. Greatest Common Divisor - Least Common Multiple

3 Greatest Commo Divisor - Least Commo Multiple Defiitio 31: The greatest commo divisor of two atural umbers a ad b is the largest atural umber c which divides both a ad b We deote the greatest commo gcd

### Lecture 5: Span, linear independence, bases, and dimension

Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;

### Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

### Lecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k.

18.409 A Algorithmist s Toolkit September 17, 009 Lecture 3 Lecturer: Joatha Keler Scribe: Adre Wibisoo 1 Outlie Today s lecture covers three mai parts: Courat-Fischer formula ad Rayleigh quotiets The

### BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

### Research Article Sign Data Derivative Recovery

Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov

### LECTURE 13: Cross-validation

LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

### The Stable Marriage Problem

The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

### Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series

8 Fourier Series Our aim is to show that uder reasoable assumptios a give -periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series

### Solving equations. Pre-test. Warm-up

Solvig equatios 8 Pre-test Warm-up We ca thik of a algebraic equatio as beig like a set of scales. The two sides of the equatio are equal, so the scales are balaced. If we add somethig to oe side of the

### Factors of sums of powers of binomial coefficients

ACTA ARITHMETICA LXXXVI.1 (1998) Factors of sums of powers of biomial coefficiets by Neil J. Cali (Clemso, S.C.) Dedicated to the memory of Paul Erdős 1. Itroductio. It is well ow that if ( ) a f,a = the

### CHAPTER 3 THE TIME VALUE OF MONEY

CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all

### Class Meeting # 16: The Fourier Transform on R n

MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

### Building Blocks Problem Related to Harmonic Series

TMME, vol3, o, p.76 Buildig Blocks Problem Related to Harmoic Series Yutaka Nishiyama Osaka Uiversity of Ecoomics, Japa Abstract: I this discussio I give a eplaatio of the divergece ad covergece of ifiite

### Chapter 5: Inner Product Spaces

Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

### Infinite Sequences and Series

CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

### A Guide to the Pricing Conventions of SFE Interest Rate Products

A Guide to the Pricig Covetios of SFE Iterest Rate Products SFE 30 Day Iterbak Cash Rate Futures Physical 90 Day Bak Bills SFE 90 Day Bak Bill Futures SFE 90 Day Bak Bill Futures Tick Value Calculatios

### Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

### Chapter 7: Confidence Interval and Sample Size

Chapter 7: Cofidece Iterval ad Sample Size Learig Objectives Upo successful completio of Chapter 7, you will be able to: Fid the cofidece iterval for the mea, proportio, ad variace. Determie the miimum

### Section 11.3: The Integral Test

Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

### SEQUENCES AND SERIES

Chapter 9 SEQUENCES AND SERIES Natural umbers are the product of huma spirit. DEDEKIND 9.1 Itroductio I mathematics, the word, sequece is used i much the same way as it is i ordiary Eglish. Whe we say

### 1 Correlation and Regression Analysis

1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

### Engineering 323 Beautiful Homework Set 3 1 of 7 Kuszmar Problem 2.51

Egieerig 33 eautiful Homewor et 3 of 7 Kuszmar roblem.5.5 large departmet store sells sport shirts i three sizes small, medium, ad large, three patters plaid, prit, ad stripe, ad two sleeve legths log

### 5: Introduction to Estimation

5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

### FOUNDATIONS OF MATHEMATICS AND PRE-CALCULUS GRADE 10

FOUNDATIONS OF MATHEMATICS AND PRE-CALCULUS GRADE 10 [C] Commuicatio Measuremet A1. Solve problems that ivolve liear measuremet, usig: SI ad imperial uits of measure estimatio strategies measuremet strategies.

### A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length

Joural o Satisfiability, Boolea Modelig ad Computatio 1 2005) 49-60 A Faster Clause-Shorteig Algorithm for SAT with No Restrictio o Clause Legth Evgey Datsi Alexader Wolpert Departmet of Computer Sciece

### Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

### Entropy of bi-capacities

Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

### Systems Design Project: Indoor Location of Wireless Devices

Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 698-5295 Email: bcm1@cec.wustl.edu Supervised

### Department of Computer Science, University of Otago

Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

### 1. C. The formula for the confidence interval for a population mean is: x t, which was

s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

### Finding the circle that best fits a set of points

Fidig the circle that best fits a set of poits L. MAISONOBE October 5 th 007 Cotets 1 Itroductio Solvig the problem.1 Priciples............................... Iitializatio.............................

INFINITE SERIES KEITH CONRAD. Itroductio The two basic cocepts of calculus, differetiatio ad itegratio, are defied i terms of limits (Newto quotiets ad Riema sums). I additio to these is a third fudametal