# 1 The Gaussian channel

Save this PDF as:

Size: px
Start display at page:

Download "1 The Gaussian channel"

## Transcription

1 ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise. We will derive the famous capacity formula. The Gaussia chael Suppose we sed iformatio over a chael that is subjected to additive white Gaussia oise. The the output is Y i = X i + Z i Y i is the chael output, X i is the chael iput, ad Z i is zero-mea Gaussia with variace N: Z i N (0, N). This is differet from chael models we saw before, i that the output ca take o a cotiuum of values. This is also a good model for a variety of practical commuicatio chaels. We will assume that there is a costrait o the iput power. If we have a iput codeword (x, x 2,..., x ), we will assume that the average power is costraied so that x 2 i P Let is cosider the probability of error for biary trasmissio. Suppose that we ca sed either + P or P over the chael. The receiver looks at the received sigal amplitude ad determies the sigal trasmitted usig a threshold test. The P e = 2 P (Y < 0 X = + P ) + 2 P (Y > 0 X = P ) = 2 P (Z < P X = + P ) + 2 P (Z > P X = P ) = P (Z > P ) = e x2 /2N dx P 2πN = Q( P/N) = Φ( P/N) or Defiitio costrait is Q(x) = e x2 /2 dx 2π x Φ(x) = 2π x e x2 /t dx The iformatio capacity of the Gaussia chael with power C = max I(X; Y ). p(x):ex 2 P

2 ECE 77: Lecture 0 The Gaussia chael 2 We ca compute this as follows: I(X; Y ) = h(y ) h(y X) = h(y ) h(x + Z X) = h(y ) h(z X) = h(y ) h(z) 2 log 2πe(P + N) log 2πeN 2 = log( + P/N) 2 sice EY 2 = P + N ad the Gaussia is the maximum-etropy distributio for a give variace. So C = log( + P/N), 2 bits per chael use. The maximum is obtaied whe X is Gaussia distributed. (How do we make the iput distributio look Gaussia?) Defiitio 2 A (M, ) code for the Gaussia chael with power costrait P cosists of the followig:. A idex set {, 2,..., M} 2. A ecodig fuctio x : {,..., M} X, which maps a iput idex ito a sequece that is elemets log, x (), x (2),..., x (M), such that the average power costraits is satisfied: for w =, 2,..., M. (x i (w)) 2 P 3. A decodig fuctio g : Y {, 2,..., M}. Defiitio 3 A rate R is said to be achievable for a a Gaussia chael with a power costrait P if there exists a sequece of (2 R, ) codes with codewords satisfyig the power costrait such that the maximal probability of error λ () 0. The capacity of the chael is the supremum of the achievable rates. Theorem The capacity of a Gaussia chael with power costrait P ad oise variace N is C = ( 2 log + P ) bits per trasmissio. N Geometric plausibility For a codeword of legth, the received vector (i space) is ormally distributed with mea equal to the true codeword. With high probability, the received vector is cotaied i sphere about the mea of radius (N + ɛ). Why? Because with high probability, the vector falls withi oe stadard deviatio away from the mea i each directio, ad the total distace away is the Euclidea sum: E[z 2 + z z 2 ] = N.

3 ECE 77: Lecture 0 The Gaussia chael 3 This is the square of the expected distace withi which we expect to fall. If we assig everythig withi this sphere to the give codeword, we misdetect oly if we fall outside this codeword. Other codewords will have other spheres, each with radius approximately (N + ɛ). The received vectors a limited i eergy by P, so they all must lie i a sphere of radius (P + N). The umber of (approximately) oitersectig decodig spheres is therefore umber of spheres volume of sphere i -space with radius r = (P + N) volume of sphere i -space with radius r = (N + ɛ) The volume of a sphere of radius r i space is proportioal to r. Substitutig i this fact we get umber of spheres ((P + N))/2 ((N + ɛ)) /2 2 2 (+ P N ) Proof We will follow essetially the same steps as before.. First we geerate a codebook at radom. This time we geerate the codebook accordig to the Gaussia distributio: let X i (w), i =, 2,..., be the code sequece correspodig to iput idex w, each X i (w) is selected at radom i.i.d. accordig to N (0, P ɛ). (With high probability, this has average power P.) The codebook is kow by both trasmitter ad receiver. 2. Ecode as described above. 3. The receiver gets a Y, ad looks at the list of codewords {X (w)} ad searches for oe which is joitly typical with the received vector. If there is oly oe such vector, it is declared as the trasmitted vector. If there is more tha oe such vector, a error is declared. A error is also declared if the chose codeword does ot satisfy the power costrait. For the probability of error, assume w.o.l.o.g. that codeword is set: Defie the followig evets: Y = X () + Z E 0 = { Xi 2 () > P } (the evet that the codeword exceeds the power costrait) ad The probability of error is the E i = {(X (i), Y ) is i A () ɛ } P (E) = P (E 0 E c E 2 E 3 E 2 R) 2 R P (E 0 ) + P (E) c + P (E i ) i=2 uio boud By LLN, P (E 0 ) 0. By joit AEP, P (E c ) 0, so P (E c ) ɛ for sufficietly large. By the code geeratio process, X () ad X (i) are idepedet, so are

4 ECE 77: Lecture 0 The Gaussia chael 4 Y ad X (i), i. So the probability that X () ad Y are joitly typical is 2 (I(X;Y ) 3ɛ) by joit AEP. So 2 R e ɛ + ɛ + P () i=2 (I(X;Y ) 3ɛ) 2 (I(X;Y ) 3ɛ) 2ɛ + (2 R )2 = 2ɛ + 2 R 2 (I(X;Y ) 3ɛ) 3ɛ for sufficietly large, if R < I(X; Y ) 3ɛ. This gives the average probability of error: we the go through the same kids of argumets as before to coclude that the maximum probability of error also must go to zero. The coverse is that rate R > C are ot achievable, or, equivaletly, that if P () e 0 the it must be that R C. Proof The proof starts with Fao s iequality: H(W Y ) + RP () e ɛ = ad ɛ 0 as. The proof is a strig of iequalities: + RP () e = ɛ R = H(W ) = I(W ; Y ) + H(W Y ) uiform W ; defiitio of I I(W ; Y ) + ɛ Fao s iequality = h(y ) h(y X ) + ɛ = h(y ) h(z ) + ɛ h(y i ) h(z ) + ɛ = h(y i ) h(z i ) + ɛ 2 log 2πe(P i + N) 2 log 2πeN + ɛ etropies of Y ad Z; power costrait = 2 log( + P i/n) + ɛ ( ) = log( + P i /N) + ɛ log( + P i /N) + ɛ Jese s 2 log( + P/N) + ɛ. Dividig through by, R 2 log( + P/N) + ɛ.

5 ECE 77: Lecture 0 The Gaussia chael 5 2 Bad-limited chaels We ow come to the first time i the book the iformatio is actually carried by a time-waveform, istead of a radom variable. We will cosider trasmissio over a bad-limited chael (such as a phoe chael). A key result is the samplig theorem: Theorem 2 If f(t) is badlimited to W Hz, the the fuctio is completely determied by samples of the fuctio take every 2W secods apart. This is the classical Nyquist samplig theorem. However, Shao s ame is also attached to it, sice he provided a proof ad used it. A represetatio of the fuctio f(t) is f(t) = sic(t) = f( 2W ) sic(t 2W ) si(2πw t) 2πW t From this theorem, we coclude (the dimesioality theorem) that a badlimited fuctio has oly 2W degrees of freedom per secod. For a sigal which has most of the eergy i badwidth W ad most of the eergy i a time T, the there are about 2W T degrees of freedom, ad the time- ad bad-limited fuctio ca be represeted usig 2W T orthogoal basis fuctios, kow as the prolate spheroidal fuctios. We ca view bad- ad timelimited fuctios as vectors i a 2T W dimesioal vector space. Assume that the oise power-spectral desity of the chael is N 0 /2. The the oise power is (N 0 /2)(2W ) = N 0 W. Over the time iterval of T secods, the eergy per sample (per chael use) is Use this iformatio i the capacity: P T 2W T = P 2W. C = 2 log( + P ) bits per chael use N = 2 log( + P ) bits per chael use. N 0 W There are 2W samples each secod (chael uses), so the capacity is or C = (2W ) 2 log( + P N 0 W ) bits/secod C = W log( + P N 0 W ) This is the famous ad key result of iformatio theory. As W, we have to do a little calculus to fid that C = P N 0 log 2 e bits per secod.

6 ECE 77: Lecture 0 The Gaussia chael 6 This is iterestig: eve with ifiite badwidth, the capacity is ot ifiite, but grows liearly with the power. Example For a phoe chael, take W = 3300 Hz. If the SNR is P/N 0 W = 40dB = 0000, we get If P/W N 0 = 20dB = 00 we get C = bits per secod. C = 2972 bits/secod. (The book is dated.) We caot do better tha capacity! 3 Kuh-Tucker Coditios Before proceedig with the ext sectio, we eed a result from costraied optimizatio theory kow as the Kuh-Tucker coditio. Suppose we are miimizig some covex objective fuctio L(x), subject to a costrait mi L(x) f(x) 0. Let the optimal value of x be x 0. The either the costrait is iactive, i which case we get L = 0 x0 or, if the costrait is active, it must be the case that the objective fuctio icreases for all admissible values of x: L 0 x A A is the set of admissible values, for which f y 0. (Thik about what happes if this is ot the case.) Thus, or sg L We ca create a ew objective fuctio so the ecessary coditios become = sg f L + λ f = 0 λ 0. () J(x, λ) = L(x) + λf(x), J = 0

7 ECE 77: Lecture 0 The Gaussia chael 7 ad f(x) 0 λ { 0 f(y) = 0 costrait is active = 0 f(y) < 0 costrait is iactive. For a vector variable x, the the coditio () meas: L is parallel to f ad poitig i opposite directios, L is iterpreted as the gradiet. I words, what coditio () says is: the gradiet of L with respect to x at a miimum must be poited i such a way that decrease of L ca oly come by violatig the costraits. Otherwise, we could decrease L further. This is the essece of the Kuh-Tucker coditio. 4 Parallel Gaussia chaels Parallel Gaussia chaels are used to model badlimited chaels with a o-flat frequecy respose. We assume we have k Gaussia chaels, Y j = X j + Z j, j =, 2,..., k. Z j N (0, N j ) ad the chaels are idepedet. The total power used is costraied: E k Xj 2 P. j= Oe questio we might ask is: how do we distribute the power across the k chaels to get maximum throughput. We ca fid the maximum mutual iformatio (the iformatio chael capacity) as I(X,..., X k ; Y,..., Y k ) = h(y,..., Y k ) h(y,..., Y k X,..., X k ) = h(y,..., Y k ) h(z,..., Z k ) k = h(y,..., Y k ) h(z i ) k h(y i ) h(z i ) i 2 log( + P i/n i ) Equality is obtaied whe the Xs are idepedet ormally distributed. We wat to distribute the power available amog the various chaels, subject to ot exceedig the power costrait: J(P,..., P k ) = i 2 log( + P i N i ) + λ k P i

8 ECE 77: Lecture 0 The Gaussia chael 8 with a side costrait (ot show) that P i 0. Differetial w.r.t. P j to obtai P j + N j + λ 0. with equality oly if all the costraits are iactive. After some fiddlig, we obtai P j = ν N j (sice λ is a costat). However, we must also have P j 0, so we must esure that we do t violate that if N j > ν. Thus, we let P j = (ν N j ) + ad ν is chose so that (x) + = { x x 0 0 x < 0 (ν N i ) + = P Draw picture; explai water fillig. 5 Chaels with colored Gaussia oise We will exted the results of the previous sectio ow to chaels with o-white Gaussia oise. Let K z be the covariace of the oise K x the covariace of the iput, with the iput costraied by EXi 2 P i which is the same as We ca write tr(k X) P. I(X,..., X ; Y,..., Y ) = h(y,..., Y ) h(z,..., Z ) h(y,..., Y ) 2 log((2πe) K x + K z ) Now how do we choose K x to maximize K x + K z, subject to the power costrait? Let the K z = QΛQ T K x + K z = K x + QΛQ T = Q Q T K x Q + Λ Q T = Q T K x Q + Λ = A + λ

9 ECE 77: Lecture 0 The Gaussia chael 9 A = Q T K x Q. Observe that tr(a) = tr(q T K x Q) = tr(q T QK x ) = tr(k x ) So we wat to maximize A + Λ subject to tr(a) P. The key is to use a iequality, i this case Hadamard s iequality. Hadamard s iequality follows directly from the coditioig reduces etropy theorem: Let X N (0, K). The ad Substitutig i ad simplifyig gives h(x,..., X ) h(x i ). h(x) = 2 log(2πe) K h(x i ) = 2 log(2πe)k ii K i K ii with equality iff K is diagoal. Gettig back to our problem, A + Λ i (A ii + Λ ii ) with equality iff A is diagoal. We have A ii P (the power costrait), ad A ii 0. As before, we take ν is chose so that i A ii = (ν λ i ) + Aii = P. Now we wat to geeralize to a cotiuous time system. For a chael with AWGN ad covariace matrix K () Z, the covariace is Toeplitz. If the chael oise process is statioary, the the covariace matrix is Toeplitz, ad the eigevalues of the covariace matrix ted to a limit as. The desity of the eigevalues o the real lie teds to the power spectrum of the stochastic process. That is, if K ij = K i j are the autocorrelatio values ad the power spectrum is the S(ω) = F[r k ] λ + λ λ M lim M M = π S(ω)dω. 2π π I this case, the water fillig traslates to water fillig i the spectral domai. The capacity of the chael with oise spectrum N(f) ca be show to be (ν N(f))+ C = log( + )df 2 N(f) ν is chose so that (ν N(f)) + df = P

### Convexity, Inequalities, and Norms

Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

### Maximum Likelihood Estimators.

Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

### SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

### Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

### Overview of some probability distributions.

Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

### University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

### Properties of MLE: consistency, asymptotic normality. Fisher information.

Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

### In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

### 0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

### Normal Distribution.

Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

### Section 11.3: The Integral Test

Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

### I. Chi-squared Distributions

1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

### 1. C. The formula for the confidence interval for a population mean is: x t, which was

s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

### Hypothesis testing. Null and alternative hypotheses

Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

### Confidence Intervals. CI for a population mean (σ is known and n > 30 or the variable is normally distributed in the.

Cofidece Itervals A cofidece iterval is a iterval whose purpose is to estimate a parameter (a umber that could, i theory, be calculated from the populatio, if measuremets were available for the whole populatio).

More information

### Lecture 4: Cheeger s Inequality

Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

More information

### 1 Computing the Standard Deviation of Sample Means

Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

### Theorems About Power Series

Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

More information

### Sampling Distribution And Central Limit Theorem

() Samplig Distributio & Cetral Limit Samplig Distributio Ad Cetral Limit Samplig distributio of the sample mea If we sample a umber of samples (say k samples where k is very large umber) each of size,

More information

### Chapter 5: Inner Product Spaces

Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

More information

### Confidence Intervals for One Mean

Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

### Sequences and Series

CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

### Lecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k.

18.409 A Algorithmist s Toolkit September 17, 009 Lecture 3 Lecturer: Joatha Keler Scribe: Adre Wibisoo 1 Outlie Today s lecture covers three mai parts: Courat-Fischer formula ad Rayleigh quotiets The

More information

### Chapter 7 Methods of Finding Estimators

Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

### CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 8

CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 8 GENE H GOLUB 1 Positive Defiite Matrices A matrix A is positive defiite if x Ax > 0 for all ozero x A positive defiite matrix has real ad positive

More information

### Infinite Sequences and Series

CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

### Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

### Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

### Soving Recurrence Relations

Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

### Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

### UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

### Determining the sample size

Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors

More information

### Universal coding for classes of sources

Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

### , a Wishart distribution with n -1 degrees of freedom and scale matrix.

UMEÅ UNIVERSITET Matematisk-statistiska istitutioe Multivariat dataaalys D MSTD79 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multivariat dataaalys D, 5 poäg.. Assume that

More information

### 4.3. The Integral and Comparison Tests

4.3. THE INTEGRAL AND COMPARISON TESTS 9 4.3. The Itegral ad Compariso Tests 4.3.. The Itegral Test. Suppose f is a cotiuous, positive, decreasig fuctio o [, ), ad let a = f(). The the covergece or divergece

More information

### Department of Computer Science, University of Otago

Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

### CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

### Math C067 Sampling Distributions

Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

More information

### Modified Line Search Method for Global Optimization

Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

### THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

More information

### Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

### Center, Spread, and Shape in Inference: Claims, Caveats, and Insights

Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the

More information

### Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series

8 Fourier Series Our aim is to show that uder reasoable assumptios a give -periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series

More information

### 5 Boolean Decision Trees (February 11)

5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

### FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. 1. Powers of a matrix

FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. Powers of a matrix We begi with a propositio which illustrates the usefuless of the diagoalizatio. Recall that a square matrix A is diogaalizable if

More information

### Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown

Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about

More information

### The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles

The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio

More information

### Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

### 5: Introduction to Estimation

5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

### Plug-in martingales for testing exchangeability on-line

Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

More information

### Confidence Intervals

Cofidece Itervals Cofidece Itervals are a extesio of the cocept of Margi of Error which we met earlier i this course. Remember we saw: The sample proportio will differ from the populatio proportio by more

More information

### CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

### Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find

1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.

More information

### Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

### The Stable Marriage Problem

The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

### Output Analysis (2, Chapters 10 &11 Law)

B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

### Lecture 5: Span, linear independence, bases, and dimension

Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;

More information

### Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is

0_0605.qxd /5/05 0:45 AM Page 470 470 Chapter 6 Additioal Topics i Trigoometry 6.5 Trigoometric Form of a Complex Number What you should lear Plot complex umbers i the complex plae ad fid absolute values

More information

### LECTURE 13: Cross-validation

LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

More information

### Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

### SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

### Domain 1 - Describe Cisco VoIP Implementations

Maual ONT (642-8) 1-800-418-6789 Domai 1 - Describe Cisco VoIP Implemetatios Advatages of VoIP Over Traditioal Switches Voice over IP etworks have may advatages over traditioal circuit switched voice etworks.

More information

### Asymptotic Growth of Functions

CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

### Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics

Chair for Network Architectures ad Services Istitute of Iformatics TU Müche Prof. Carle Network Security Chapter 2 Basics 2.4 Radom Number Geeratio for Cryptographic Protocols Motivatio It is crucial to

More information

### A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design

A Combied Cotiuous/Biary Geetic Algorithm for Microstrip Atea Desig Rady L. Haupt The Pesylvaia State Uiversity Applied Research Laboratory P. O. Box 30 State College, PA 16804-0030 haupt@ieee.org Abstract:

More information

### Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

### Basic Elements of Arithmetic Sequences and Series

MA40S PRE-CALCULUS UNIT G GEOMETRIC SEQUENCES CLASS NOTES (COMPLETED NO NEED TO COPY NOTES FROM OVERHEAD) Basic Elemets of Arithmetic Sequeces ad Series Objective: To establish basic elemets of arithmetic

More information

### Chapter 14 Nonparametric Statistics

Chapter 14 Noparametric Statistics A.K.A. distributio-free statistics! Does ot deped o the populatio fittig ay particular type of distributio (e.g, ormal). Sice these methods make fewer assumptios, they

More information

### Practice Problems for Test 3

Practice Problems for Test 3 Note: these problems oly cover CIs ad hypothesis testig You are also resposible for kowig the samplig distributio of the sample meas, ad the Cetral Limit Theorem Review all

More information

### Overview on S-Box Design Principles

Overview o S-Box Desig Priciples Debdeep Mukhopadhyay Assistat Professor Departmet of Computer Sciece ad Egieerig Idia Istitute of Techology Kharagpur INDIA -721302 What is a S-Box? S-Boxes are Boolea

More information

### 3 Basic Definitions of Probability Theory

3 Basic Defiitios of Probability Theory 3defprob.tex: Feb 10, 2003 Classical probability Frequecy probability axiomatic probability Historical developemet: Classical Frequecy Axiomatic The Axiomatic defiitio

More information

### One-sample test of proportions

Oe-sample test of proportios The Settig: Idividuals i some populatio ca be classified ito oe of two categories. You wat to make iferece about the proportio i each category, so you draw a sample. Examples:

More information

### GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.

GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all

More information

### Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

### BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

More information

### MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

### Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals

Overview Estimatig the Value of a Parameter Usig Cofidece Itervals We apply the results about the sample mea the problem of estimatio Estimatio is the process of usig sample data estimate the value of

More information

### An Efficient Polynomial Approximation of the Normal Distribution Function & Its Inverse Function

A Efficiet Polyomial Approximatio of the Normal Distributio Fuctio & Its Iverse Fuctio Wisto A. Richards, 1 Robi Atoie, * 1 Asho Sahai, ad 3 M. Raghuadh Acharya 1 Departmet of Mathematics & Computer Sciece;

More information

### 1 Correlation and Regression Analysis

1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

More information

### CS103X: Discrete Structures Homework 4 Solutions

CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible six-figure salaries i whole dollar amouts are there that cotai at least

More information

### Basic Measurement Issues. Sampling Theory and Analog-to-Digital Conversion

Theory ad Aalog-to-Digital Coversio Itroductio/Defiitios Aalog-to-digital coversio Rate Frequecy Aalysis Basic Measuremet Issues Reliability the extet to which a measuremet procedure yields the same results

More information

### NATIONAL SENIOR CERTIFICATE GRADE 12

NATIONAL SENIOR CERTIFICATE GRADE MATHEMATICS P EXEMPLAR 04 MARKS: 50 TIME: 3 hours This questio paper cosists of 8 pages ad iformatio sheet. Please tur over Mathematics/P DBE/04 NSC Grade Eemplar INSTRUCTIONS

More information

### Multi-server Optimal Bandwidth Monitoring for QoS based Multimedia Delivery Anup Basu, Irene Cheng and Yinzhe Yu

Multi-server Optimal Badwidth Moitorig for QoS based Multimedia Delivery Aup Basu, Iree Cheg ad Yizhe Yu Departmet of Computig Sciece U. of Alberta Architecture Applicatio Layer Request receptio -coectio

More information

### Statistical inference: example 1. Inferential Statistics

Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

More information

### MEI Structured Mathematics. Module Summary Sheets. Statistics 2 (Version B: reference to new book)

MEI Mathematics i Educatio ad Idustry MEI Structured Mathematics Module Summary Sheets Statistics (Versio B: referece to ew book) Topic : The Poisso Distributio Topic : The Normal Distributio Topic 3:

More information

### Interference Alignment and the Generalized Degrees of Freedom of the X Channel

Iterferece Aligmet ad the Geeralized Degrees of Freedom of the X Chael Chiachi Huag, Viveck R. Cadambe, Syed A. Jafar Electrical Egieerig ad Computer Sciece Uiversity of Califoria Irvie Irvie, Califoria,

More information

### AP Calculus BC 2003 Scoring Guidelines Form B

AP Calculus BC Scorig Guidelies Form B The materials icluded i these files are iteded for use by AP teachers for course ad exam preparatio; permissio for ay other use must be sought from the Advaced Placemet

More information

### A probabilistic proof of a binomial identity

A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

### Your organization has a Class B IP address of 166.144.0.0 Before you implement subnetting, the Network ID and Host ID are divided as follows:

Subettig Subettig is used to subdivide a sigle class of etwork i to multiple smaller etworks. Example: Your orgaizatio has a Class B IP address of 166.144.0.0 Before you implemet subettig, the Network

More information

### Descriptive Statistics

Descriptive Statistics We leared to describe data sets graphically. We ca also describe a data set umerically. Measures of Locatio Defiitio The sample mea is the arithmetic average of values. We deote

More information

### Listing terms of a finite sequence List all of the terms of each finite sequence. a) a n n 2 for 1 n 5 1 b) a n for 1 n 4 n 2

74 (4 ) Chapter 4 Sequeces ad Series 4. SEQUENCES I this sectio Defiitio Fidig a Formula for the th Term The word sequece is a familiar word. We may speak of a sequece of evets or say that somethig is

More information

### Partial Di erential Equations

Partial Di eretial Equatios Partial Di eretial Equatios Much of moder sciece, egieerig, ad mathematics is based o the study of partial di eretial equatios, where a partial di eretial equatio is a equatio

More information

### 1. MATHEMATICAL INDUCTION

1. MATHEMATICAL INDUCTION EXAMPLE 1: Prove that for ay iteger 1. Proof: 1 + 2 + 3 +... + ( + 1 2 (1.1 STEP 1: For 1 (1.1 is true, sice 1 1(1 + 1. 2 STEP 2: Suppose (1.1 is true for some k 1, that is 1

More information

### Repeating Decimals are decimal numbers that have number(s) after the decimal point that repeat in a pattern.

5.5 Fractios ad Decimals Steps for Chagig a Fractio to a Decimal. Simplify the fractio, if possible. 2. Divide the umerator by the deomiator. d d Repeatig Decimals Repeatig Decimals are decimal umbers

More information

### Chapter 7: Confidence Interval and Sample Size

Chapter 7: Cofidece Iterval ad Sample Size Learig Objectives Upo successful completio of Chapter 7, you will be able to: Fid the cofidece iterval for the mea, proportio, ad variace. Determie the miimum

More information

### On the Capacity of Hybrid Wireless Networks

O the Capacity of Hybrid ireless Networks Beyua Liu,ZheLiu +,DoTowsley Departmet of Computer Sciece Uiversity of Massachusetts Amherst, MA 0002 + IBM T.J. atso Research Ceter P.O. Box 704 Yorktow Heights,

More information

### Research Method (I) --Knowledge on Sampling (Simple Random Sampling)

Research Method (I) --Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact

More information

### .04. This means \$1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

More information

### PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

### Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean

1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.

More information