Class 5: Kalman Filter
|
|
- Shonda Hodges
- 7 years ago
- Views:
Transcription
1 Class 5: Kalman Filter Macroeconometrics - Spring 2011 Jacek Suda, BdF and PSE April 11, 2011
2 Outline Outline: 1 Prediction Error Decomposition 2 State-space Form 3 Deriving the Kalman Filter See Kim and Nelson, Chapter 3; Hamilton, Chapter 13
3 Notation Denote: {Y t } covariance-stationary process, e.g. ARMA(p, q), Ω t information available at time t, Y t+1 t forecast of Y t+1 based on Ω t. In our simple model it is Y y+1 given Y t.
4 Linear Projection Linear projection: Ŷ t+1 t = α X t = α 1 X 1t α p X pt, where E[(Y t+1 α X t ) X it ] = 0, i = 1,..., p. p moments conditions ensure that error is orthogonal to any information in Ω t: forecast errors are uncorrelated with past information. Result: The minimum MSE linear forecast of Y t+1 is linear projection.
5 ARMA Models Solve Wold form Y t µ = ψ(l)ε t, ε t WN ψ(l) = ψ j L j, ψ 0 = 1, ψj 2 < j=0 j=0 Y t+s = µ + ε t+s + ψ 1 ε t+s ψ s ε t + ψ s+1 ε t , Ŷ t+1 t = µ + ψ s ε t + ψ s+1 ε t the last line is for information till time t and E t [ε t+i ] = 0, i > 0.
6 ARMA Models MSE(Ŷ t+s t, Y t+s ) = E[(ε t + ψε t+s ψ s 1 ε t+1 ) 2 ] = σ 2 (1 + ψ ψ ψ 2 s 1) < var(y t+s ). But, We are better off with linear projection than with unconditional variance. lim s σ2 s ψk 2 = var(y t ). k=0 Upper limit for uncertainty is as high as the unconditional variance.
7 Kalman Filter Forecasts based on Wold form assume infinite number of observations. We don t have them in reality. Kalman filter calculates linear projections for finite number of observations, exact finite sample forecast, allow for exact MLE of ARMA models based on Prediction Error Decomposition.
8 Normal Distribution Joint Normality: y 1. y T = ỹ TT 1 N(µ T 1, Ω T T ), Since it is covariance stationary process, each Y t has the same mean and variance, ω 11 = σ 2 = ω 22 = ω TT. ω 11 ω 12 ω 1T γ 0 γ 1... γ T 1 Ω = ω 21 ω = γ 1 γ 0 ω jj = γ 0,...., as ω ij = γ i j, i > j. γ T 1 γ 0 ω T1 ω TT The likelihood function: L( θ ỹ T ) = (2π) T 2 det(ω) 1 2 e 1 2 (ỹt µ) Ω 1 (ỹ T µ).
9 Factorization For large T, Ω might be large and difficult to invert. Since Ω is positive definite symmetric matrix then there exists a unique, triangular factorization of Ω, Ω = AfA, where f T T = A T T = f f , f t > 0 t diagonal matrix 0 f T 1 0 a a T1 a T2... 1
10 Likelihood The likelihood function can be rewritten as: L( θ ỹ T ) = (2π) T 2 det(afa ) 1 2 e 1 2 (ỹt µ) (AfA ) 1 (ỹ T µ) Define η = A 1 (ỹ T µ)(prediction error). where Aη = (ỹ T µ). Since A is lower-triangular matrix with 1s along the principal diagonal, η 1 = y 1 µ η 2 = y 2 µ a 11η 1 η 3 = y 3 µ a 21η 1 a 22η 2. T 1 η T = y T µ i=1 a Tiη T 1
11 Likelihood Also, since A is lower triangular with 1s along the principal diagonal, det(a) = 1 Then, det(afa) = det(a) det(f ) det(a ) = det(f ). L( θ ỹ T ) = (2π) T 2 det(f 1 ) 1 2 e 1 2 η (f 1 ) 1 η = T ( ) 1 e 1 η t 2 2 ft, 2πft t=1 where η t is t th element of η T 1 = prediction error y t ŷ t t 1, t 1 ŷ t t 1 = a t,iy i, i = 2, 3,..., T, i=1 where a t,i is (t, i) th element of A 1.
12 Kalman Filter Note: Given y t N(µ, Ω), η t Ω t 1 N(0, f t ), where f t is an (t, t) diagonal element of f matrix, ln L = 1 2 T ln(2πf t ) 1 2 t=1 T t=1 η 2 t f t, since η t N and independent of each other. The Kalman filter recursively calculates linear projection of y t on past information Ω t 1 for any model that can be cast in state-space form. Kalman filter: for any structure it solves for linear prediction.
13 Measurement (Observation) Equation General form that encompasses a wide variety od models. 1 Measurement (Observation) Equation Represent the static relationship between observed variables (data) and unobserved state variables. y t = H t β t + Az t + e t, where y t denotes observed data, β t is a state vector that captures the dynamics, z t is exogenous, observed variables for example, lagged values of y t but also other data, and e t is an error term, e t N(0, R). The existence of the state vector makes this representation not a simple linear model.
14 Transition (State) Equation 2 Transition (State) Equation Captures the dynamics in the system, causes the system to go on and on. β t = µ + Fβ t 1 + v t, where µ is a vector of constants, F is the transition matrix, and v t is an error vector, v t N(0, Q). Like AR(1) but in vector/matrix form.
15 Transition (State) Equation β t = µ + Fβ t 1 + v t, The state vector has and AR(1) kind of representation. Describes evolution of state vector. These state vectors can be unobservable. Transition equation can be used to get information about the unobservable, conditioning on data which is observable (Bayesian).
16 Error terms Error terms: e t N(0, R), v t N(0, Q), where R, Q are var-cov matrices and E[e t v τ ] = 0, t, τ Restrictive assumption The model can be represented in a way that is not very restrictive. Even with E[e t v τ ] 0 we can estimate the model with (modified) Kalman Filter but it becomes more complicated. The normality assumption might not be good as always but... It allows to use MLE.
17 Examples: AR(p) It applies to a very wide variety of time-series models. Consider an AR(p) process State equation y t µ = φ 1 (y t 1 µ) φ p (y t p µ) + ε t E(ε τ ε t ) = σ 2 for t = τ y t µ y t 1 µ. y t p+1 µ = φ 1 φ 2... φ p 1 φ p y t 1 µ y t 2 µ. y t p µ + ε t 0. 0 Observation equation y t = µ + [ ] y t 1 µ y t 2 µ. y t p µ.
18 Examples: ARMA(1,1) ARMA(1,1): Set µ = 0, y t = φy t 1 + ε t + θε t 1, ε t N(0, σ 2 ). There might be more than one way to represent a model in a state-space form. There might be differences in efficiency between different ways.
19 Examples: ARMA(1,1) State equation: The general form Put β t = β t = Fβ t 1 + v t. [ ] [ ] yt yt 1 β ε t 1 = t ε t 1 Put y t = φy t 1 + θε t 1 + ε t in a matrix notation: [ ] [ ] [ ] yt φ θ yt 1 = + ε t 0 0 ε t 1 β t F β t 1 [ ] σ 2 σ and v t N(0, Q), Q = 2 σ 2 σ 2. y t observable, ε t unobservable, forecast error [ εt ε t ] v t,
20 Examples: ARMA(1,1) Observation equations: y t = [ 1 0 ] [ yt ε t y t H β t ] no exogenous variables: A = 0, also R = 0. y t = Hβ t for this case (ARMA(1,1)). The parameters φ, θ, σ 2 are captured in F, Q matrices. The Kalman Filter will estimate them. For KF what goes in β t doesn t matter. Only parameters F, Q, H, R will matter. The state vector is now defined by F, Q, H, and the observations.
21 ARMA(1,1): Alternative Representation A more elegant ( i.e. easier for computation) representation. Log notation (alternative representation for ARMA(1,1)) (1 φl)y t = (1 + θl)ε t y t = (1 φl) 1 (1 + θl)ε t y t = (1 + θl)(1 φl) 1 ε t. Define x t = (1 φl) 1 ε t (1 φl)x t = ε t, (x t is AR(1), not observed) x t φx t 1 = ε t Then, y t = (1 + θl)x t y t = x t + θx t 1. So y t is a linear combination of 2 unobservable AR(1) processes, x t and X t 1.
22 ARMA(1,1): State-Space Observation equation (all randomness in the state equation) where y t = Hβ t, y t = [ 1 0 ] [ x t x t 1 ] Inside H there are parameters to be estimated. A = 0, no exogenous, R = 0 as the observable equation is just the identity (no randomness of e t.
23 ARMA(1,1): State-Space State equation so [ xt x t 1 ] β t = [ ] φ F v t N(0, Q), Q = So φ is in F, θ in H, and σ 2 in Q. [ xt 1 x t 2 ] β t 1 + [ σ ]. [ εt ] 0, v t Given F, Q, H, A, R and data (y t s), use Kalman Filter to find prediction error decomposition of joint likelihood for ỹ T = (y 1,... y T ), given by L(θ, φ, σ 2 ỹ T ). (exact likelihood)
24 Kalman Filter Kalman filter: purpose: to make inference about unobservable given the observable, application: signal extraction in engineering, economics: don t know the parameters F, Q, H and want to estimate them. State-space form ME: Measurement (Observation) equation: SE: Transition (State) equation: y t = Hβ t + e t, e t N(0, R) β t = µ + Fβ t 1 + v t, v t N(0, Q), E[e t v τ ] = 0.
25 Mean of β 1 β t is a random variable it might be unobservable and no data for it, it is normal random variable as it is sum of normal variables, v t N. Conditional mean β t Ω t 1 N (E[β t Ω t 1 ], var(β t Ω t 1 )) E[β t Ω t 1 ] = β t t 1, conditional expectations. We may not know what β s are. If we have information about its distribution, we can calculate mean, variance, etc. β t 1 may be not observable: take expectations of it E[β t Ω t 1 ] β t t 1 = µ + FE[β t 1 Ω t 1 ] + 0 β t t 1 = µ + Fβ t 1 t 1, In AR(1): E[y t] = µ + φe[y t 1], last term is observable.
26 Variance of β Conditional variance Recall Var(β t Ω t 1 ) P t t 1 = E[(β t β t t 1 )(β t β t t 1 ) ]. var(ax) = a 2 var(x), a scalar, x random vector. Two sources of randomness (variation) for β t : 1 v t is a random variable, 2 β t 1 is also random so there might be difference between β t 1 and β t t 1, there may not be equal to each other. P t t 1 = F P t 1 t 1 F + Q, where P t t 1, uncertainty about β t equals sum of uncertainty about β t 1, P t 1 t 1, and uncertainty about v t. Note: cov(β t 1, v t ) = 0.
27 Kalman Filter 2 y t is a random variable. Now, we have data on y t. We have some joint density of y t, β t and some prior. Using data we get posterior of β t. We want to make inference for β t which we don t observe. We see y t which is related to beta t. We make inferences on β t by observing joint density (distribution) of ys and βs (Bayesian view).
28 Distribution of y t Distribution of y t given state-space y t Ω t 1 N(E[y t Ω t 1 ], var(y t Ω t 1 ), Conditional mean E[y t Ω t 1 ] y t t 1 = Hβ t t Conditional variance var(y t Ω t 1 ) f t t 1 = HP t t 1 H + Q, since we don t know β t. Note: cov(hβ t, e t ) = 0 because E[v t e t ] = 0. If E[v te t] 0 we will add another term in the var(y t Ω t 1) capturing that.
29 Joint Distribution Covariance between β t and y t : cov(y t, β t Ω t 1 ) = P t t 1 H, as cov(hβ t + e t, β t ) = cov(hβ t, β t ) + cov(e t, β t ) = cov(β t, β t )H + 0. Then, the joint distribution for y t and β t is joint normal: ([ ] [ ]) β t y t Ω βt t 1 Pt t 1 P t 1 N, t t 1 H Hβ t t 1 P t t 1 H. f t t 1
30 Kalman Filter Two steps of Kalman Filter : (a) Prediction, (b) Given y t updating inference on β t. Definition Given β 0 0, P 0 0, Kalman Filter solves the following six equations for i = 1,..., T Prediction of y t, β t (1) β t t 1 = µ + Fβ t 1 t 1, (2) P t t 1 = F P t 1 t 1 F + Q, Forecast error: Variance of forecast error: (3) η t t 1 y t y t t 1 = y t Hβ t t 1, (4) f t t 1 = H P t t 1 H + R Updating of y t, β t (5) β t t = β t t 1 + κ tη t t 1, (6) P t t = P t t 1 κ thp t t 1, κ t P t t 1 H f 1 t t 1 Kalman gain.
31 Kalman Filter β 0 0, P 0 0, are equal to unconditional mean and variance, and reflect prior beliefs. Equation (5) is a linear combination of previous guess and forecast error. (5) β t t = β t t 1 + κ t η t t 1, (6) P t t = P t t 1 κ t HP t t 1, κ t P t t 1 H f 1 t t 1 Kalman gain. The Kalman gain depends on the relationship between y t and β t since P t t 1 H = cov(β t, y t) and f 1 t t 1 is the precision of the forecast error. The bigger the variance of forecast error the smaller the Kalman gain and less weight put to updating. Equation (6) measures conditional variance. Since we observe y t the uncertainty declines.
32 Kalman Gain The stronger the covariance between y t and β t, the more we will update when we see high forecast error. If the relationship is weaker, we don t put much weight as probably it is not driven by β t. The weight depends on the variance of forecast error: if f 1 big, put high weight on that observations. Once we have η t t 1, f t t 1, we can do MLE after constructing the joint likelihood of prediction error decomposition.
Time Series Analysis III
Lecture 12: Time Series Analysis III MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Time Series Analysis III 1 Outline Time Series Analysis III 1 Time Series Analysis III MIT 18.S096 Time Series Analysis
More informationTime Series Analysis 1. Lecture 8: Time Series Analysis. Time Series Analysis MIT 18.S096. Dr. Kempthorne. Fall 2013 MIT 18.S096
Lecture 8: Time Series Analysis MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Time Series Analysis 1 Outline Time Series Analysis 1 Time Series Analysis MIT 18.S096 Time Series Analysis 2 A stochastic
More informationChapter 1. Vector autoregressions. 1.1 VARs and the identi cation problem
Chapter Vector autoregressions We begin by taking a look at the data of macroeconomics. A way to summarize the dynamics of macroeconomic data is to make use of vector autoregressions. VAR models have become
More informationState Space Time Series Analysis
State Space Time Series Analysis p. 1 State Space Time Series Analysis Siem Jan Koopman http://staff.feweb.vu.nl/koopman Department of Econometrics VU University Amsterdam Tinbergen Institute 2011 State
More informationFinancial TIme Series Analysis: Part II
Department of Mathematics and Statistics, University of Vaasa, Finland January 29 February 13, 2015 Feb 14, 2015 1 Univariate linear stochastic models: further topics Unobserved component model Signal
More information1 Short Introduction to Time Series
ECONOMICS 7344, Spring 202 Bent E. Sørensen January 24, 202 Short Introduction to Time Series A time series is a collection of stochastic variables x,.., x t,.., x T indexed by an integer value t. The
More informationImpulse Response Functions
Impulse Response Functions Wouter J. Den Haan University of Amsterdam April 28, 2011 General definition IRFs The IRF gives the j th -period response when the system is shocked by a one-standard-deviation
More informationEstimating an ARMA Process
Statistics 910, #12 1 Overview Estimating an ARMA Process 1. Main ideas 2. Fitting autoregressions 3. Fitting with moving average components 4. Standard errors 5. Examples 6. Appendix: Simple estimators
More informationUnivariate and Multivariate Methods PEARSON. Addison Wesley
Time Series Analysis Univariate and Multivariate Methods SECOND EDITION William W. S. Wei Department of Statistics The Fox School of Business and Management Temple University PEARSON Addison Wesley Boston
More informationy t by left multiplication with 1 (L) as y t = 1 (L) t =ª(L) t 2.5 Variance decomposition and innovation accounting Consider the VAR(p) model where
. Variance decomposition and innovation accounting Consider the VAR(p) model where (L)y t = t, (L) =I m L L p L p is the lag polynomial of order p with m m coe±cient matrices i, i =,...p. Provided that
More informationTime Series Analysis
Time Series Analysis Andrea Beccarini Center for Quantitative Economics Winter 2013/2014 Andrea Beccarini (CQE) Time Series Analysis Winter 2013/2014 1 / 143 Introduction Objectives Time series are ubiquitous
More informationChapter 4: Vector Autoregressive Models
Chapter 4: Vector Autoregressive Models 1 Contents: Lehrstuhl für Department Empirische of Wirtschaftsforschung Empirical Research and und Econometrics Ökonometrie IV.1 Vector Autoregressive Models (VAR)...
More information3.1 Stationary Processes and Mean Reversion
3. Univariate Time Series Models 3.1 Stationary Processes and Mean Reversion Definition 3.1: A time series y t, t = 1,..., T is called (covariance) stationary if (1) E[y t ] = µ, for all t Cov[y t, y t
More informationTime Series Analysis
Time Series Analysis Forecasting with ARIMA models Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 2012 Alonso and García-Martos (UC3M-UPM)
More informationUnderstanding and Applying Kalman Filtering
Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance
More informationSYSTEMS OF REGRESSION EQUATIONS
SYSTEMS OF REGRESSION EQUATIONS 1. MULTIPLE EQUATIONS y nt = x nt n + u nt, n = 1,...,N, t = 1,...,T, x nt is 1 k, and n is k 1. This is a version of the standard regression model where the observations
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationMultivariate time series analysis is used when one wants to model and explain the interactions and comovements among a group of time series variables:
Multivariate Time Series Consider n time series variables {y 1t },...,{y nt }. A multivariate time series is the (n 1) vector time series {Y t } where the i th row of {Y t } is {y it }.Thatis,for any time
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationSales forecasting # 2
Sales forecasting # 2 Arthur Charpentier arthur.charpentier@univ-rennes1.fr 1 Agenda Qualitative and quantitative methods, a very general introduction Series decomposition Short versus long term forecasting
More informationSystem Identification for Acoustic Comms.:
System Identification for Acoustic Comms.: New Insights and Approaches for Tracking Sparse and Rapidly Fluctuating Channels Weichang Li and James Preisig Woods Hole Oceanographic Institution The demodulation
More informationBasics of Statistical Machine Learning
CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar
More informationE 4101/5101 Lecture 8: Exogeneity
E 4101/5101 Lecture 8: Exogeneity Ragnar Nymoen 17 March 2011 Introduction I Main references: Davidson and MacKinnon, Ch 8.1-8,7, since tests of (weak) exogeneity build on the theory of IV-estimation Ch
More information11. Time series and dynamic linear models
11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd
More informationProbability and Random Variables. Generation of random variables (r.v.)
Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly
More informationTrend and Seasonal Components
Chapter 2 Trend and Seasonal Components If the plot of a TS reveals an increase of the seasonal and noise fluctuations with the level of the process then some transformation may be necessary before doing
More informationTime Series Analysis
Time Series Analysis Lecture Notes for 475.726 Ross Ihaka Statistics Department University of Auckland April 14, 2005 ii Contents 1 Introduction 1 1.1 Time Series.............................. 1 1.2 Stationarity
More informationThe VAR models discussed so fare are appropriate for modeling I(0) data, like asset returns or growth rates of macroeconomic time series.
Cointegration The VAR models discussed so fare are appropriate for modeling I(0) data, like asset returns or growth rates of macroeconomic time series. Economic theory, however, often implies equilibrium
More informationEigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
More informationLecture 2: ARMA(p,q) models (part 3)
Lecture 2: ARMA(p,q) models (part 3) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC) Univariate time series Sept.
More informationChapter 5. Analysis of Multiple Time Series. 5.1 Vector Autoregressions
Chapter 5 Analysis of Multiple Time Series Note: The primary references for these notes are chapters 5 and 6 in Enders (2004). An alternative, but more technical treatment can be found in chapters 10-11
More informationGaussian Conjugate Prior Cheat Sheet
Gaussian Conjugate Prior Cheat Sheet Tom SF Haines 1 Purpose This document contains notes on how to handle the multivariate Gaussian 1 in a Bayesian setting. It focuses on the conjugate prior, its Bayesian
More informationMAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX
MAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX KRISTOFFER P. NIMARK The next section derives the equilibrium expressions for the beauty contest model from Section 3 of the main paper. This is followed by
More informationTime Series in Mathematical Finance
Instituto Superior Técnico (IST, Portugal) and CEMAT cnunes@math.ist.utl.pt European Summer School in Industrial Mathematics Universidad Carlos III de Madrid July 2013 Outline The objective of this short
More informationChapter 3: The Multiple Linear Regression Model
Chapter 3: The Multiple Linear Regression Model Advanced Econometrics - HEC Lausanne Christophe Hurlin University of Orléans November 23, 2013 Christophe Hurlin (University of Orléans) Advanced Econometrics
More informationQuadratic forms Cochran s theorem, degrees of freedom, and all that
Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us
More informationAnalysis of Bayesian Dynamic Linear Models
Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main
More informationLecture 8: Signal Detection and Noise Assumption
ECE 83 Fall Statistical Signal Processing instructor: R. Nowak, scribe: Feng Ju Lecture 8: Signal Detection and Noise Assumption Signal Detection : X = W H : X = S + W where W N(, σ I n n and S = [s, s,...,
More informationTime series Forecasting using Holt-Winters Exponential Smoothing
Time series Forecasting using Holt-Winters Exponential Smoothing Prajakta S. Kalekar(04329008) Kanwal Rekhi School of Information Technology Under the guidance of Prof. Bernard December 6, 2004 Abstract
More information1 Teaching notes on GMM 1.
Bent E. Sørensen January 23, 2007 1 Teaching notes on GMM 1. Generalized Method of Moment (GMM) estimation is one of two developments in econometrics in the 80ies that revolutionized empirical work in
More informationLinear regression methods for large n and streaming data
Linear regression methods for large n and streaming data Large n and small or moderate p is a fairly simple problem. The sufficient statistic for β in OLS (and ridge) is: The concept of sufficiency is
More informationLecture 5 Least-squares
EE263 Autumn 2007-08 Stephen Boyd Lecture 5 Least-squares least-squares (approximate) solution of overdetermined equations projection and orthogonality principle least-squares estimation BLUE property
More informationRegression Analysis. Regression Analysis MIT 18.S096. Dr. Kempthorne. Fall 2013
Lecture 6: Regression Analysis MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Regression Analysis 1 Outline Regression Analysis 1 Regression Analysis MIT 18.S096 Regression Analysis 2 Multiple Linear
More informationCentre for Central Banking Studies
Centre for Central Banking Studies Technical Handbook No. 4 Applied Bayesian econometrics for central bankers Andrew Blake and Haroon Mumtaz CCBS Technical Handbook No. 4 Applied Bayesian econometrics
More informationEE 570: Location and Navigation
EE 570: Location and Navigation On-Line Bayesian Tracking Aly El-Osery 1 Stephen Bruder 2 1 Electrical Engineering Department, New Mexico Tech Socorro, New Mexico, USA 2 Electrical and Computer Engineering
More informationForecasting Geographic Data Michael Leonard and Renee Samy, SAS Institute Inc. Cary, NC, USA
Forecasting Geographic Data Michael Leonard and Renee Samy, SAS Institute Inc. Cary, NC, USA Abstract Virtually all businesses collect and use data that are associated with geographic locations, whether
More informationIn this paper we study how the time-series structure of the demand process affects the value of information
MANAGEMENT SCIENCE Vol. 51, No. 6, June 25, pp. 961 969 issn 25-199 eissn 1526-551 5 516 961 informs doi 1.1287/mnsc.15.385 25 INFORMS Information Sharing in a Supply Chain Under ARMA Demand Vishal Gaur
More informationTime Series Analysis
Time Series Analysis Autoregressive, MA and ARMA processes Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 212 Alonso and García-Martos
More informationThe Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More information**BEGINNING OF EXAMINATION** The annual number of claims for an insured has probability function: , 0 < q < 1.
**BEGINNING OF EXAMINATION** 1. You are given: (i) The annual number of claims for an insured has probability function: 3 p x q q x x ( ) = ( 1 ) 3 x, x = 0,1,, 3 (ii) The prior density is π ( q) = q,
More informationBayes and Naïve Bayes. cs534-machine Learning
Bayes and aïve Bayes cs534-machine Learning Bayes Classifier Generative model learns Prediction is made by and where This is often referred to as the Bayes Classifier, because of the use of the Bayes rule
More informationUnivariate Time Series Analysis; ARIMA Models
Econometrics 2 Spring 25 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Outline of the Lecture () Introduction to univariate time series analysis. (2) Stationarity. (3) Characterizing
More information3. Regression & Exponential Smoothing
3. Regression & Exponential Smoothing 3.1 Forecasting a Single Time Series Two main approaches are traditionally used to model a single time series z 1, z 2,..., z n 1. Models the observation z t as a
More informationIntroduction to Matrix Algebra
Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary
More informationUnivariate Time Series Analysis; ARIMA Models
Econometrics 2 Fall 25 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Univariate Time Series Analysis We consider a single time series, y,y 2,..., y T. We want to construct simple
More informationOn the Efficiency of Competitive Stock Markets Where Traders Have Diverse Information
Finance 400 A. Penati - G. Pennacchi Notes on On the Efficiency of Competitive Stock Markets Where Traders Have Diverse Information by Sanford Grossman This model shows how the heterogeneous information
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part X Factor analysis Whenwehavedatax (i) R n thatcomesfromamixtureofseveral Gaussians, the EM algorithm can be applied to fit a mixture model. In this setting, we usually
More informationMaster s thesis tutorial: part III
for the Autonomous Compliant Research group Tinne De Laet, Wilm Decré, Diederik Verscheure Katholieke Universiteit Leuven, Department of Mechanical Engineering, PMA Division 30 oktober 2006 Outline General
More informationSections 2.11 and 5.8
Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and
More information8 Square matrices continued: Determinants
8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More informationA Model for Hydro Inow and Wind Power Capacity for the Brazilian Power Sector
A Model for Hydro Inow and Wind Power Capacity for the Brazilian Power Sector Gilson Matos gilson.g.matos@ibge.gov.br Cristiano Fernandes cris@ele.puc-rio.br PUC-Rio Electrical Engineering Department GAS
More informationForecast covariances in the linear multiregression dynamic model.
Forecast covariances in the linear multiregression dynamic model. Catriona M Queen, Ben J Wright and Casper J Albers The Open University, Milton Keynes, MK7 6AA, UK February 28, 2007 Abstract The linear
More informationChapter 10: Basic Linear Unobserved Effects Panel Data. Models:
Chapter 10: Basic Linear Unobserved Effects Panel Data Models: Microeconomic Econometrics I Spring 2010 10.1 Motivation: The Omitted Variables Problem We are interested in the partial effects of the observable
More informationTime Series Analysis
Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Identification of univariate time series models, cont.:
More informationAnalysis of algorithms of time series analysis for forecasting sales
SAINT-PETERSBURG STATE UNIVERSITY Mathematics & Mechanics Faculty Chair of Analytical Information Systems Garipov Emil Analysis of algorithms of time series analysis for forecasting sales Course Work Scientific
More informationi=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by
Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a p-dimensional parameter
More informationDynamic Linear Models with R
Giovanni Petris, Sonia Petrone, Patrizia Campagnoli Dynamic Linear Models with R SPIN Springer s internal project number, if known Monograph August 10, 2007 Springer Berlin Heidelberg NewYork Hong Kong
More informationUnit 18 Determinants
Unit 18 Determinants Every square matrix has a number associated with it, called its determinant. In this section, we determine how to calculate this number, and also look at some of the properties of
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationIntroduction to Time Series Analysis. Lecture 1.
Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series
More informationPrinciple Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression
Principle Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression Saikat Maitra and Jun Yan Abstract: Dimension reduction is one of the major tasks for multivariate
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More informationMultivariate Normal Distribution
Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues
More informationCITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION
No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August
More informationOctober 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix
Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,
More informationα = u v. In other words, Orthogonal Projection
Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v
More informationMath 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.
Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(
More informationSolution to Homework 2
Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
More informationInternet Appendix to CAPM for estimating cost of equity capital: Interpreting the empirical evidence
Internet Appendix to CAPM for estimating cost of equity capital: Interpreting the empirical evidence This document contains supplementary material to the paper titled CAPM for estimating cost of equity
More informationChristfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic
More informationTime Series Analysis in Economics. Klaus Neusser
Time Series Analysis in Economics Klaus Neusser May 26, 2015 Contents I Univariate Time Series Analysis 3 1 Introduction 1 1.1 Some examples.......................... 2 1.2 Formal definitions.........................
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationModeling and Performance Evaluation of Computer Systems Security Operation 1
Modeling and Performance Evaluation of Computer Systems Security Operation 1 D. Guster 2 St.Cloud State University 3 N.K. Krivulin 4 St.Petersburg State University 5 Abstract A model of computer system
More informationStatistics in Retail Finance. Chapter 6: Behavioural models
Statistics in Retail Finance 1 Overview > So far we have focussed mainly on application scorecards. In this chapter we shall look at behavioural models. We shall cover the following topics:- Behavioural
More informationTIME SERIES ANALYSIS
TIME SERIES ANALYSIS L.M. BHAR AND V.K.SHARMA Indian Agricultural Statistics Research Institute Library Avenue, New Delhi-0 02 lmb@iasri.res.in. Introduction Time series (TS) data refers to observations
More informationLecture 4: Seasonal Time Series, Trend Analysis & Component Model Bus 41910, Time Series Analysis, Mr. R. Tsay
Lecture 4: Seasonal Time Series, Trend Analysis & Component Model Bus 41910, Time Series Analysis, Mr. R. Tsay Business cycle plays an important role in economics. In time series analysis, business cycle
More informationReview Jeopardy. Blue vs. Orange. Review Jeopardy
Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David
More informationWhat s New in Econometrics? Lecture 8 Cluster and Stratified Sampling
What s New in Econometrics? Lecture 8 Cluster and Stratified Sampling Jeff Wooldridge NBER Summer Institute, 2007 1. The Linear Model with Cluster Effects 2. Estimation with a Small Number of Groups and
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More information2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system
1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables
More information1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
More informationITSM-R Reference Manual
ITSM-R Reference Manual George Weigt June 5, 2015 1 Contents 1 Introduction 3 1.1 Time series analysis in a nutshell............................... 3 1.2 White Noise Variance.....................................
More informationANALYZING INVESTMENT RETURN OF ASSET PORTFOLIOS WITH MULTIVARIATE ORNSTEIN-UHLENBECK PROCESSES
ANALYZING INVESTMENT RETURN OF ASSET PORTFOLIOS WITH MULTIVARIATE ORNSTEIN-UHLENBECK PROCESSES by Xiaofeng Qian Doctor of Philosophy, Boston University, 27 Bachelor of Science, Peking University, 2 a Project
More information