How to Derive Mean and Mean Square Error for an Estimator?
|
|
- Maurice Parks
- 7 years ago
- Views:
Transcription
1 How to Derive Mean and Mean Square Error for an Estimator? Hing Cheung So ( 蘇慶祥 ) Department of Electronic Engineering City University of Hong Kong H. C. So Page 1 Nov. 2013
2 Outline Introduction Mean and Mean Square Error Formulas for Scalar Examples for Scalar Estimation Mean and Mean Square Error Formulas for Vector Examples for Vector Estimation List of References H. C. So Page 2 Nov. 2013
3 Introduction What is Estimation? Estimation refers to accurately finding the values of parameters of interest from the observed data which consist of two components, viz., signal and noise A generic model for estimation of a scalar is: where is the observation vector, signal is a known function of and noise is an additive random process The estimation problem is to find given H. C. So Page 3 Nov. 2013
4 Why Estimation is Needed? Many science and engineering problems can be boiled down to parameter estimation: Radar Radar system transmits an electromagnetic pulse. It is reflected by an aircraft, causing an echo to be received H. C. So Page 4 Nov. 2013
5 Time delay is round trip propagation time of radar pulse. If we know, can be obtained as, is speed of light H. C. So Page 5 Nov. 2013
6 Mobile Communications If we know one-way propagation time of the signal traveling between mobile station and base station (BS), then the target position can be obtained using three BSs H. C. So Page 6 Nov. 2013
7 Speech Processing vowel of "a" time (s) For a voiced speech, it can be modeled as a periodic signal and it is important to estimate its pitch or fundamental frequency for analysis. H. C. So Page 7 Nov. 2013
8 Image Processing Estimation of the position and orientation of an object from a camera image is useful when using a robot to pick it up, e.g., bomb-disposal Biomedical Engineering Estimation the heart rate of a fetus and the difficulty is that the measurements are corrupted by the mother s heart beat as well Seismology Estimation of the underground distance of an oil deposit based on sound reflection due to the different densities of oil and rock layers Astronomy Estimation of the periods of orbits H. C. So Page 8 Nov. 2013
9 How to Perform Estimation? Least squares (LS) and maximum likelihood (ML) are two standard estimation approaches Consider the model of The LS estimator does not require the probability density function (PDF) of, and its estimate is obtained by minimizing a sum of squared error: where H. C. So Page 9 Nov. 2013
10 To produce the ML estimator, the PDF of is required Assuming that is a zero-mean Gaussian noise, the PDF of the observed vector, which is parameterized by, is where The ML estimate is: When is white with variance, the PDF reduces to ML estimate is reduced to LS solution H. C. So Page 10 Nov. 2013
11 How to Assess Estimators? Two standard performance measures for assessing accuracy of an estimator are bias and mean square error (MSE): and It is desired that or, indicating that the estimator is unbiased, and MSE is as small as possible For an unbiased estimator, MSE is equal to variance: In general: H. C. So Page 11 Nov. 2013
12 Consider a simple problem of estimating a DC level from: where has mean 0 and variance We easily suggest three estimators: H. C. So Page 12 Nov. 2013
13 It is easy to show: ; ; ; is the best among the three because it has zero bias and minimum variance Is optimum? How to compute bias and MSE for more general cases? H. C. So Page 13 Nov. 2013
14 Cramér-Rao Lower Bound (CRLB) CRLB is performance bound in terms of minimum achievable variance provided by any unbiased estimators Its derivation requires knowledge of the noise PDF and the PDF must have closed-form Although there are other variance bounds, CRLB is simplest Suppose the PDF of where, is The CRLB for can be obtained in two steps: Compute the Fisher information matrix CRLB for is the entry of, H. C. So Page 14 Nov. 2013
15 has the form of: H. C. So Page 15 Nov. 2013
16 Consider with zero-mean white Gaussian noise: That is, is the optimum estimator for H. C. So Page 16 Nov. 2013
17 Mean and Mean Square Error Formulas for Scalar Recall the signal model: Suppose the scalar is estimated by minimizing a differentiable cost function constructed from, : This implies H. C. So Page 17 Nov. 2013
18 At small estimation error conditions, Tayler series expansion yields: is close to. Applying If is sufficiently smooth around, then Hence Similarly, H. C. So Page 18 Nov. 2013
19 Note: When is a quadratic function: and When, is an unbiased estimate of For unbiased estimator: H. C. So Page 19 Nov. 2013
20 Examples for Scalar Estimation For simplicity, we assume that the noise is white Gaussian process with variance DC Level Estimation Recall the model: Using the LS approach, the cost function to be minimized is H. C. So Page 20 Nov. 2013
21 To apply the bias and MSE formulas, we compute: and Hence: and which align with previous analysis H. C. So Page 21 Nov. 2013
22 Time-Difference-of-Arrival Estimation The simplest model is to estimate the time-difference-ofarrival between two signals: where, and are independent zero-mean white Gaussian variables with, It is clear that is most similar to. As a result, can be estimated by maximizing the cross-correlation between and : H. C. So Page 22 Nov. 2013
23 However, is generally not an integer and thus is a continuous function of Using the convolution theorem, has the form of Applying the bias and MSE formulas, we obtain: and which is also the CRLB H. C. So Page 23 Nov. 2013
24 Frequency Estimation of a Complex Sinusoid The signal model is: where, and A conventional approach for estimating periodogram peak: is to search the Applying the bias and MSE formulas, we obtain: and which is also the CRLB H. C. So Page 24 Nov. 2013
25 Frequency Estimation of a Real Sinusoid The signal model is: where, and According to the linear prediction property: A LS cost function for estimating is then: H. C. So Page 25 Nov. 2013
26 The LS estimate of is: Hence the frequency estimate is which is known as the modified covariance method H. C. So Page 26 Nov. 2013
27 Applying the bias and MSE formulas, we obtain: and if is sufficiently large Hence where H. C. So Page 27 Nov. 2013
28 With tedious calculation, we have Since We have: H. C. So Page 28 Nov. 2013
29 Mean and Mean Square Error Formulas for Vector For estimation of a vector from minimizing, the formulas are generalized as follows: and where is the gradient vector and is the Hessian matrix: H. C. So Page 29 Nov. 2013
30 As a result, Similarly, the covariance matrix is: The MSE of is given by entry of H. C. So Page 30 Nov. 2013
31 Examples for Vector Estimation Estimation of a Linear Model The linear data model is: where is known, is unknown vector, and Employing, the weighted LS cost function is: Applying the bias and MSE formulas, we obtain: and These align with the best linear unbiased estimator (BLUE) H. C. So Page 31 Nov. 2013
32 Parameter Estimation of a Real Sinusoid The signal model is: where, and, while is a white Gaussian process with variance According to ML or LS, we construct: H. C. So Page 32 Nov. 2013
33 Applying the bias and MSE formulas, we obtain: which is the inverse of the Fisher information matrix That is, the estimator is optimum H. C. So Page 33 Nov. 2013
34 Localization using Range Measurements Consider positioning of a source at sensors at known coordinates, by If we have the one-way propagation time measurements, they can be easily converted to ranges: where and is white The ML or LS cost function is H. C. So Page 34 Nov. 2013
35 To determine the bias and MSE, the steps include: because Similarly, H. C. So Page 35 Nov. 2013
36 As a result, With tedious calculation, we have which is the inverse of the Fisher information matrix That is, the estimator is optimum H. C. So Page 36 Nov. 2013
37 Apart from the nonlinear approach, can be linearized: where Hence the signal model is now linear: where H. C. So Page 37 Nov. 2013
38 For sufficiently small noise conditions, we have H. C. So Page 38 Nov. 2013
39 The weighted LS cost function to be minimized is and the estimate is Applying the bias and MSE formulas, we obtain: MSEs of and are given by and entries of H. C. So Page 39 Nov. 2013
40 To achieve higher accuracy, the information of should be utilized, which results in a constrained optimization problem: The solution can be derived using the method of Lagrange multipliers To analyze the performance, the constrained problem can be converted to an unconstrained one by putting the relation of into : H. C. So Page 40 Nov. 2013
41 Applying the bias and MSE formulas, we obtain: which is the inverse of the Fisher information matrix That is, the estimator is also optimum H. C. So Page 41 Nov. 2013
42 List of References [1] H.C. So, Y.T. Chan, K.C. Ho and Y. Chen, Simple formulas for bias and mean square error computation, IEEE Signal Processing Magazine, vol.30, no.4, pp , Jul [2] S.M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice-Hall, NJ: Englewood Cliffs, 1993 [3] V.H. MacDonald and P.M. Schultheiss, Optimum passive bearing estimation in a spatially incoherent noise environment,' Journal of the Acoustical Society of America, vol.46, no.1, pt.1, pp.37-43, Jul [4] H.C. So, Y.T. Chan and F.K.W. Chan, Closed-form formulae for optimum time difference of arrival based localization, IEEE Transactions on Signal Processing, H. C. So Page 42 Nov. 2013
43 vol.56, no.6, pp , Jun [5] K.W.K. Lui and H.C. So, Performance of modified covariance estimator for a single real tone, IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol.e90-a, no.9, pp , Sep [6] K.W. Cheung, H.C. So, W.-K. Ma and Y.T. Chan, Least squares algorithms for time-of-arrival based mobile location, IEEE Transactions on Signal Processing, vol.52, no.4, pp , Apr [7] H.C. So, Source Localization: Algorithms and Analysis, Handbook of Position Location: Theory, Practice and Advances, Chapter 2, S.A. Zekavat and M. Buehrer, Eds., Wiley-IEEE Press, 2011 H. C. So Page 43 Nov. 2013
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationProbability and Random Variables. Generation of random variables (r.v.)
Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly
More informationLecture 8: Signal Detection and Noise Assumption
ECE 83 Fall Statistical Signal Processing instructor: R. Nowak, scribe: Feng Ju Lecture 8: Signal Detection and Noise Assumption Signal Detection : X = W H : X = S + W where W N(, σ I n n and S = [s, s,...,
More informationLeast-Squares Intersection of Lines
Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
More informationExploiting A Constellation of Narrowband RF Sensors to Detect and Track Moving Targets
Exploiting A Constellation of Narrowband RF Sensors to Detect and Track Moving Targets Chris Kreucher a, J. Webster Stayman b, Ben Shapo a, and Mark Stuff c a Integrity Applications Incorporated 900 Victors
More informationSignal Detection. Outline. Detection Theory. Example Applications of Detection Theory
Outline Signal Detection M. Sami Fadali Professor of lectrical ngineering University of Nevada, Reno Hypothesis testing. Neyman-Pearson (NP) detector for a known signal in white Gaussian noise (WGN). Matched
More informationThe Calculation of G rms
The Calculation of G rms QualMark Corp. Neill Doertenbach The metric of G rms is typically used to specify and compare the energy in repetitive shock vibration systems. However, the method of arriving
More informationThe equivalence of logistic regression and maximum entropy models
The equivalence of logistic regression and maximum entropy models John Mount September 23, 20 Abstract As our colleague so aptly demonstrated ( http://www.win-vector.com/blog/20/09/the-simplerderivation-of-logistic-regression/
More informationReview Jeopardy. Blue vs. Orange. Review Jeopardy
Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?
More informationPa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classifica6on
Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classifica6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationSignal Detection C H A P T E R 14 14.1 SIGNAL DETECTION AS HYPOTHESIS TESTING
C H A P T E R 4 Signal Detection 4. SIGNAL DETECTION AS HYPOTHESIS TESTING In Chapter 3 we considered hypothesis testing in the context of random variables. The detector resulting in the minimum probability
More informationCCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York
BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not
More informationKristine L. Bell and Harry L. Van Trees. Center of Excellence in C 3 I George Mason University Fairfax, VA 22030-4444, USA kbell@gmu.edu, hlv@gmu.
POSERIOR CRAMÉR-RAO BOUND FOR RACKING ARGE BEARING Kristine L. Bell and Harry L. Van rees Center of Excellence in C 3 I George Mason University Fairfax, VA 22030-4444, USA bell@gmu.edu, hlv@gmu.edu ABSRAC
More informationAutomatic Detection of Emergency Vehicles for Hearing Impaired Drivers
Automatic Detection of Emergency Vehicles for Hearing Impaired Drivers Sung-won ark and Jose Trevino Texas A&M University-Kingsville, EE/CS Department, MSC 92, Kingsville, TX 78363 TEL (36) 593-2638, FAX
More informationIncreasing for all. Convex for all. ( ) Increasing for all (remember that the log function is only defined for ). ( ) Concave for all.
1. Differentiation The first derivative of a function measures by how much changes in reaction to an infinitesimal shift in its argument. The largest the derivative (in absolute value), the faster is evolving.
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationMidwest Symposium On Circuits And Systems, 2004, v. 2, p. II137-II140. Creative Commons: Attribution 3.0 Hong Kong License
Title Adaptive window selection and smoothing of Lomb periodogram for time-frequency analysis of time series Author(s) Chan, SC; Zhang, Z Citation Midwest Symposium On Circuits And Systems, 2004, v. 2,
More informationMonotonicity Hints. Abstract
Monotonicity Hints Joseph Sill Computation and Neural Systems program California Institute of Technology email: joe@cs.caltech.edu Yaser S. Abu-Mostafa EE and CS Deptartments California Institute of Technology
More informationBy choosing to view this document, you agree to all provisions of the copyright laws protecting it.
This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal
More informationEpipolar Geometry. Readings: See Sections 10.1 and 15.6 of Forsyth and Ponce. Right Image. Left Image. e(p ) Epipolar Lines. e(q ) q R.
Epipolar Geometry We consider two perspective images of a scene as taken from a stereo pair of cameras (or equivalently, assume the scene is rigid and imaged with a single camera from two different locations).
More information(Quasi-)Newton methods
(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting
More informationFinal Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones
Final Year Project Progress Report Frequency-Domain Adaptive Filtering Myles Friel 01510401 Supervisor: Dr.Edward Jones Abstract The Final Year Project is an important part of the final year of the Electronic
More informationLocation of Mobile Terminals Using Time Measurements and Survey Points
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 52, NO. 4, JULY 2003 999 Location of Mobile Terminals Using Time Measurements and Survey Points Michael McGuire, Member, IEEE, Konstantinos N. Plataniotis,
More informationAn introduction to OBJECTIVE ASSESSMENT OF IMAGE QUALITY. Harrison H. Barrett University of Arizona Tucson, AZ
An introduction to OBJECTIVE ASSESSMENT OF IMAGE QUALITY Harrison H. Barrett University of Arizona Tucson, AZ Outline! Approaches to image quality! Why not fidelity?! Basic premises of the task-based approach!
More informationName: Section Registered In:
Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are
More informationLog-Likelihood Ratio-based Relay Selection Algorithm in Wireless Network
Recent Advances in Electrical Engineering and Electronic Devices Log-Likelihood Ratio-based Relay Selection Algorithm in Wireless Network Ahmed El-Mahdy and Ahmed Walid Faculty of Information Engineering
More informationJPEG compression of monochrome 2D-barcode images using DCT coefficient distributions
Edith Cowan University Research Online ECU Publications Pre. JPEG compression of monochrome D-barcode images using DCT coefficient distributions Keng Teong Tan Hong Kong Baptist University Douglas Chai
More informationFEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL
FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint
More informationMeasuring Line Edge Roughness: Fluctuations in Uncertainty
Tutor6.doc: Version 5/6/08 T h e L i t h o g r a p h y E x p e r t (August 008) Measuring Line Edge Roughness: Fluctuations in Uncertainty Line edge roughness () is the deviation of a feature edge (as
More informationModelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic
More informationBasics of Statistical Machine Learning
CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar
More informationTime Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication
Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Thomas Reilly Data Physics Corporation 1741 Technology Drive, Suite 260 San Jose, CA 95110 (408) 216-8440 This paper
More informationMUSIC-like Processing of Pulsed Continuous Wave Signals in Active Sonar Experiments
23rd European Signal Processing Conference EUSIPCO) MUSIC-like Processing of Pulsed Continuous Wave Signals in Active Sonar Experiments Hock Siong LIM hales Research and echnology, Singapore hales Solutions
More informationUnderstanding and Applying Kalman Filtering
Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding
More informationMaximum Likelihood Estimation of ADC Parameters from Sine Wave Test Data. László Balogh, Balázs Fodor, Attila Sárhegyi, and István Kollár
Maximum Lielihood Estimation of ADC Parameters from Sine Wave Test Data László Balogh, Balázs Fodor, Attila Sárhegyi, and István Kollár Dept. of Measurement and Information Systems Budapest University
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationA Multi-Model Filter for Mobile Terminal Location Tracking
A Multi-Model Filter for Mobile Terminal Location Tracking M. McGuire, K.N. Plataniotis The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 1 King s College
More informationAdvanced Signal Processing and Digital Noise Reduction
Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New
More informationT = 1 f. Phase. Measure of relative position in time within a single period of a signal For a periodic signal f(t), phase is fractional part t p
Data Transmission Concepts and terminology Transmission terminology Transmission from transmitter to receiver goes over some transmission medium using electromagnetic waves Guided media. Waves are guided
More informationVEHICLE TRACKING USING ACOUSTIC AND VIDEO SENSORS
VEHICLE TRACKING USING ACOUSTIC AND VIDEO SENSORS Aswin C Sankaranayanan, Qinfen Zheng, Rama Chellappa University of Maryland College Park, MD - 277 {aswch, qinfen, rama}@cfar.umd.edu Volkan Cevher, James
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
More informationPrincipal components analysis
CS229 Lecture notes Andrew Ng Part XI Principal components analysis In our discussion of factor analysis, we gave a way to model data x R n as approximately lying in some k-dimension subspace, where k
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More information171:290 Model Selection Lecture II: The Akaike Information Criterion
171:290 Model Selection Lecture II: The Akaike Information Criterion Department of Biostatistics Department of Statistics and Actuarial Science August 28, 2012 Introduction AIC, the Akaike Information
More informationPattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
More informationThe Bivariate Normal Distribution
The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included
More informationAn Experimental Study of the Performance of Histogram Equalization for Image Enhancement
International Journal of Computer Sciences and Engineering Open Access Research Paper Volume-4, Special Issue-2, April 216 E-ISSN: 2347-2693 An Experimental Study of the Performance of Histogram Equalization
More informationEnhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm
1 Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm Hani Mehrpouyan, Student Member, IEEE, Department of Electrical and Computer Engineering Queen s University, Kingston, Ontario,
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher
More informationby the matrix A results in a vector which is a reflection of the given
Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that
More informationLecture 5 Least-squares
EE263 Autumn 2007-08 Stephen Boyd Lecture 5 Least-squares least-squares (approximate) solution of overdetermined equations projection and orthogonality principle least-squares estimation BLUE property
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
More informationApplying Active Queue Management to Link Layer Buffers for Real-time Traffic over Third Generation Wireless Networks
Applying Active Queue Management to Link Layer Buffers for Real-time Traffic over Third Generation Wireless Networks Jian Chen and Victor C.M. Leung Department of Electrical and Computer Engineering The
More informationTime Series and Forecasting
Chapter 22 Page 1 Time Series and Forecasting A time series is a sequence of observations of a random variable. Hence, it is a stochastic process. Examples include the monthly demand for a product, the
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David
More informationPartial Fractions. Combining fractions over a common denominator is a familiar operation from algebra:
Partial Fractions Combining fractions over a common denominator is a familiar operation from algebra: From the standpoint of integration, the left side of Equation 1 would be much easier to work with than
More informationTrend and Seasonal Components
Chapter 2 Trend and Seasonal Components If the plot of a TS reveals an increase of the seasonal and noise fluctuations with the level of the process then some transformation may be necessary before doing
More informationCITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION
No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August
More informationLinear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong
More information7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
More informationEfficient Curve Fitting Techniques
15/11/11 Life Conference and Exhibition 11 Stuart Carroll, Christopher Hursey Efficient Curve Fitting Techniques - November 1 The Actuarial Profession www.actuaries.org.uk Agenda Background Outline of
More informationDealing with large datasets
Dealing with large datasets (by throwing away most of the data) Alan Heavens Institute for Astronomy, University of Edinburgh with Ben Panter, Rob Tweedie, Mark Bastin, Will Hossack, Keith McKellar, Trevor
More informationSolutions to Math 51 First Exam January 29, 2015
Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not
More informationINDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition)
INDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition) Abstract Indirect inference is a simulation-based method for estimating the parameters of economic models. Its
More informationAutomatic Fall Detector based on Sliding Window Principle
Automatic Fall Detector based on Sliding Window Principle J. Rodriguez 1, M. Mercuri 2, P. Karsmakers 3,4, P.J. Soh 2, P. Leroux 3,5, and D. Schreurs 2 1 UPC, Div. EETAC-TSC, Castelldefels, Spain 2 KU
More informationSeveral Views of Support Vector Machines
Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min
More informationAdding Heart to Your Technology
RMCM-01 Heart Rate Receiver Component Product code #: 39025074 KEY FEATURES High Filtering Unit Designed to work well on constant noise fields SMD component: To be installed as a standard component to
More informationFOREWORD. Executive Secretary
FOREWORD The Botswana Examinations Council is pleased to authorise the publication of the revised assessment procedures for the Junior Certificate Examination programme. According to the Revised National
More informationMultivariate normal distribution and testing for means (see MKB Ch 3)
Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................
More informationSignal to Noise Instrumental Excel Assignment
Signal to Noise Instrumental Excel Assignment Instrumental methods, as all techniques involved in physical measurements, are limited by both the precision and accuracy. The precision and accuracy of a
More informationADAPTIVE ALGORITHMS FOR ACOUSTIC ECHO CANCELLATION IN SPEECH PROCESSING
www.arpapress.com/volumes/vol7issue1/ijrras_7_1_05.pdf ADAPTIVE ALGORITHMS FOR ACOUSTIC ECHO CANCELLATION IN SPEECH PROCESSING 1,* Radhika Chinaboina, 1 D.S.Ramkiran, 2 Habibulla Khan, 1 M.Usha, 1 B.T.P.Madhav,
More informationMath 215 Project (25 pts) : Using Linear Algebra to solve GPS problem
Due Thursday March 1, 2012 NAME(S): Math 215 Project (25 pts) : Using Linear Algebra to solve GPS problem 0.1 Introduction The age old question, Where in the world am I? can easily be solved nowadays by
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationEfficiency and the Cramér-Rao Inequality
Chapter Efficiency and the Cramér-Rao Inequality Clearly we would like an unbiased estimator ˆφ (X of φ (θ to produce, in the long run, estimates which are fairly concentrated i.e. have high precision.
More informationSpatial Accuracy Assessment of Digital Surface Models: A Probabilistic Approach
Spatial Accuracy Assessment of Digital Surface Models: A Probabilistic Approach André Jalobeanu Centro de Geofisica de Evora, Portugal part of the SPACEFUSION Project (ANR, France) Outline Goals and objectives
More informationMeasurement of Capacitance
Measurement of Capacitance Pre-Lab Questions Page Name: Class: Roster Number: Instructor:. A capacitor is used to store. 2. What is the SI unit for capacitance? 3. A capacitor basically consists of two
More informationMultiple Linear Regression in Data Mining
Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 Multiple
More informationTHE SELECTION OF RETURNS FOR AUDIT BY THE IRS. John P. Hiniker, Internal Revenue Service
THE SELECTION OF RETURNS FOR AUDIT BY THE IRS John P. Hiniker, Internal Revenue Service BACKGROUND The Internal Revenue Service, hereafter referred to as the IRS, is responsible for administering the Internal
More informationVision based Vehicle Tracking using a high angle camera
Vision based Vehicle Tracking using a high angle camera Raúl Ignacio Ramos García Dule Shu gramos@clemson.edu dshu@clemson.edu Abstract A vehicle tracking and grouping algorithm is presented in this work
More information1 Solving LPs: The Simplex Algorithm of George Dantzig
Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.
More informationLinear Programming. March 14, 2014
Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1
More informationLinear Algebra Notes
Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note
More informationChristfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic
More informationAP Physics 1 and 2 Lab Investigations
AP Physics 1 and 2 Lab Investigations Student Guide to Data Analysis New York, NY. College Board, Advanced Placement, Advanced Placement Program, AP, AP Central, and the acorn logo are registered trademarks
More information8. Linear least-squares
8. Linear least-squares EE13 (Fall 211-12) definition examples and applications solution of a least-squares problem, normal equations 8-1 Definition overdetermined linear equations if b range(a), cannot
More informationAu = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.
Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry
More informationProbabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014
Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about
More informationLS.6 Solution Matrices
LS.6 Solution Matrices In the literature, solutions to linear systems often are expressed using square matrices rather than vectors. You need to get used to the terminology. As before, we state the definitions
More information2DI36 Statistics. 2DI36 Part II (Chapter 7 of MR)
2DI36 Statistics 2DI36 Part II (Chapter 7 of MR) What Have we Done so Far? Last time we introduced the concept of a dataset and seen how we can represent it in various ways But, how did this dataset came
More informationADVANCED APPLICATIONS OF ELECTRICAL ENGINEERING
Development of a Software Tool for Performance Evaluation of MIMO OFDM Alamouti using a didactical Approach as a Educational and Research support in Wireless Communications JOSE CORDOVA, REBECA ESTRADA
More informationCONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Relations Between Time Domain and Frequency Domain Prediction Error Methods - Tomas McKelvey
COTROL SYSTEMS, ROBOTICS, AD AUTOMATIO - Vol. V - Relations Between Time Domain and Frequency Domain RELATIOS BETWEE TIME DOMAI AD FREQUECY DOMAI PREDICTIO ERROR METHODS Tomas McKelvey Signal Processing,
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationi=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by
Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a p-dimensional parameter
More information8.2. Solution by Inverse Matrix Method. Introduction. Prerequisites. Learning Outcomes
Solution by Inverse Matrix Method 8.2 Introduction The power of matrix algebra is seen in the representation of a system of simultaneous linear equations as a matrix equation. Matrix algebra allows us
More informationMultivariate Analysis of Variance (MANOVA): I. Theory
Gregory Carey, 1998 MANOVA: I - 1 Multivariate Analysis of Variance (MANOVA): I. Theory Introduction The purpose of a t test is to assess the likelihood that the means for two groups are sampled from the
More informationAdaptive Equalization of binary encoded signals Using LMS Algorithm
SSRG International Journal of Electronics and Communication Engineering (SSRG-IJECE) volume issue7 Sep Adaptive Equalization of binary encoded signals Using LMS Algorithm Dr.K.Nagi Reddy Professor of ECE,NBKR
More information