Outline. Random Variables. Examples. Random Variable

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Outline. Random Variables. Examples. Random Variable"

Transcription

1 Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation, convolution, correlation coefficient. Normal distribution. 1 2 s 1 S Random Variable X(.) R X : S R Function mapping the elements of the sample space S to the real line R. Equivalent Event: real value associated with the elementary event(s). Probability of the real value = sum of probabilities of the original associated elementary event(s) in the sample space. 3 Examples Example: Throw die, outcome 1-6 dots. Random Variable: maps i dots to i, i =1,, 6. Example: Measurement of any physical quantity with additive random error (noise). Example: pitch, card game, collect tricks. Probabilities of equivalent events: 4

2 Cumulative Probability Distribution Function Definition: The cumulative distribution function (CDF) of a random variable is a function defined for each real number x as follows Properties of the CDF 1. as 2. as 3. is a nondecreasing function of 5 6 Probability Density Function (pdf) Continuous random variable Nonnegative function f defined on the real line. For every real interval f X (x) x x + dx x X x dx f ( x dx P X ) x Properties of the pdf 3. First two properties follow from the axioms of probability. Integrate: 8

3 Uniform Distribution 1, a x b f X ( x) b a 0, elsewhere x a 1 x 2 b 1/(b a) x Example: Spin the Pointer Probability of value in any subinterval of is proportional to its length. Area of rectangle must be unity MATLAB >> rand(m, n) % a=0, b=1, m by n 9 10 Density & Distribution Expectation of a Random Variable Expected value or mean of. Justified by relative frequency. Discrete Continuous 11 12

4 Function of a Random Variable Properties of Expectation Expected value of the function Discrete: Continuous: Jensen s Inequality For a random variable Convex function: and a function 1 Moments moment: expectation of power First moment is the mean Second moment is the mean square Discrete Continuous 15 16

5 Variance Second moment about the mean Standard Deviation: square root of variance Discrete Continuous 17 Properties of the Variance Uncorrelated: 18 Variance Property: Proof, 19 Example: Uniform Distribution Find the mean and variance Mean: Mean Square: Variance: 20

6 Moment Generating Function L Laplace transform of pdf (also defined with Use 2-sided Laplace transform table. 21 Series Expansion and Moments Moments: 22 Characteristic Function = Fourier transform: Use Fourier transform tables Moments: Normal or Gaussian Density Symmetric about the mean. Peak value (at )= (larger for sharper peak) Mode (most likely value) = mean Standard normal distribution, zero-mean, unit variance /

7 Why is it important? Fits many physical phenomena. Central limit theorem. Completely described by mean and variance. Independent uncorrelated. Right Tail Probability Probability of exceeding a given value. >> p = normspec([-inf,1],0,1,'outside') Complementary cumulative distribution. / =Prob. of false alarm Inverse monotonically decreasing invertible. Inverse is important in some applications (signal detection: prob. of false alarm). Erf : error function Error Function Erfc: complementary error function (invertible) erf(x) erfc(x) x

8 Relation to Normal Distribution Probability Less than Upper Bound is Normal Distribution: mean zero, variance 0.5 N For negative use Density Critical Value 29 x Erf and Gaussian Density / / 30 MATLAB: Computing Probabilities (similar for Maple) >> erf(x) % Error function >> erfc(x) % Complementary error function >> 0.5*(1+erf(x/sqrt(2)) ) % St. Normal P (t < x) >> 0.5*erfc(x/sqrt(2)) % St. Normal P (t > x) >> Qinv=sqrt(2)*erfinv(1-2*P) % Inverse Q(P) 31 Example: Test Scores Test scores are normally distributed with N >> fun exp(-(x-83).^2/128)./sqrt(128*pi); >> integral(fun,83-16,83+16) % within 2 sigma ans =

9 Pseudorandom Number Generators Impulsive pdf >> rand % Uniform distribution over [0,1] >> randn % Standard normal Shifting and Scaling (also see random): >> y = sigmay*randn + ybar For Use impulse for discrete or mixed random variables. -2 f(y) F(y) Y (Y) N N Example: Half-wave rectifier Half-wave rectifier driven by noise 1 N R + R + R 0 Density/Distribution 35 36

10 Multiple Random Variables Multivariate Distributions Multivariate: vector of random variables Bivariate: 2 variables Discrete case: joint prob. = 2-dim. array x 2 A = n by 1 vector Obtain marginal prob. by adding col. or row x 20 dx 2 dx1 x 10 x 1 Generalize: Marginal pdf Marginal Distributions Example 39 Conditional Distribution Conditional probability Conditional density of given,,,,,,,,,,,,,,,,,,,,, 40

11 Bayes Rule for Random Vars. Independence,,,,, Independent,,,,,,,,,, dy Sum of Independent Random Vars. y z + d z dx z =x + y x Y Y Y 43 Independent vs. Uncorrelated Independent Uncorrelated Independent Uncorrelated. Uncorrelated Independent?? Not true, in general. True for multivariate Gaussian. 44

12 Central Limit Theorem Given n independent random variables Property of Convolutions: Convolution of a large number of positive functions is approximately Gaussian. Central Limit Theorem: Z is asymptotically Gaussian. Lim N 45 Correlation Coefficient Normalized measure of correlation between. Value between 1 and 1 Zero for uncorrelated Reduces to variance property for 46 Zero Correlation Coefficient Unity Correlation Coefficient Uncorrelated For uncorrelated 47 48

13 Range of Correlation Coefficient Proof: Quadratic in a has no real roots Discriminant: negative or zero discriminant for quadratic in a (zero for, equal roots) 49 Orthogonal Random Variables 50 Correlation and Covariance Covariance Matrix Generalization of 2 nd moment & variance to vector case. 51 Can be written in terms of variances and correlation coefficients. Diagonal for uncorrelated (& independent) variables. 52

14 Proof: Quadratic form Multivariate Normal / Generalization of normal distribution to n linearly independent random variables. If are mutually uncorrelated they are also independent. Independent/Uncorrelated If Gaussian are mutually uncorrelated they are also mutually independent. Bivariate Gaussian / / 55 56

15 Properties of Multivariate Normal Density N completely defined by and. If joint pdf is normal: Uncorrelated Independent. All marginal and conditional pdfs are normal. Linear transformation of normal vector gives a normal vector (next presentation). Conclusion Probabilistic description of random variables. Moments, characteristic function, moment generating function. Correlation and covariance. Correlated, independent, orthogonal. Normal (Gaussian) random variable References Brown & Hwang, Introduction to Random Signals and Applied Kalman Filtering, Wiley, NY, Stark & Woods, Probability and Random Processes, Prentice Hall, Upper Saddle River, NJ, R. M. Gray & L. D. Davisson, Random Processes: A mathematical Approach for Engineers, Prentice Hall, Englewood Cliffs, NJ, M. H. De Groot, M. J. Schervish, Probability & Statistics, Addison-Wesley, Boston, S. M. Kay, Fundamentals of Statistical Signal Processing: Detection Theory, Prentice Hall, A. Papoulis and S.U. Pillai, Probability, Random Variables, and Stochastic Processes, 4th Ed., McGraw Hill, Boston, MA,

Topic 4: Multivariate random variables. Multiple random variables

Topic 4: Multivariate random variables. Multiple random variables Topic 4: Multivariate random variables Joint, marginal, and conditional pmf Joint, marginal, and conditional pdf and cdf Independence Expectation, covariance, correlation Conditional expectation Two jointly

More information

SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation

SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu September 19, 2015 Outline

More information

Probability and Statistics

Probability and Statistics CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

More information

Probability and Random Variables. Generation of random variables (r.v.)

Probability and Random Variables. Generation of random variables (r.v.) Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly

More information

MULTIVARIATE PROBABILITY DISTRIBUTIONS

MULTIVARIATE PROBABILITY DISTRIBUTIONS MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined

More information

Signal Detection. Outline. Detection Theory. Example Applications of Detection Theory

Signal Detection. Outline. Detection Theory. Example Applications of Detection Theory Outline Signal Detection M. Sami Fadali Professor of lectrical ngineering University of Nevada, Reno Hypothesis testing. Neyman-Pearson (NP) detector for a known signal in white Gaussian noise (WGN). Matched

More information

Probability and statistics; Rehearsal for pattern recognition

Probability and statistics; Rehearsal for pattern recognition Probability and statistics; Rehearsal for pattern recognition Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception

More information

L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

More information

Monte Carlo Method: Probability

Monte Carlo Method: Probability John (ARC/ICAM) Virginia Tech... Math/CS 4414: The Monte Carlo Method: PROBABILITY http://people.sc.fsu.edu/ jburkardt/presentations/ monte carlo probability.pdf... ARC: Advanced Research Computing ICAM:

More information

PROBABILITY AND STATISTICS. Ma 527. 1. To teach a knowledge of combinatorial reasoning.

PROBABILITY AND STATISTICS. Ma 527. 1. To teach a knowledge of combinatorial reasoning. PROBABILITY AND STATISTICS Ma 527 Course Description Prefaced by a study of the foundations of probability and statistics, this course is an extension of the elements of probability and statistics introduced

More information

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

More information

, where f(x,y) is the joint pdf of X and Y. Therefore

, where f(x,y) is the joint pdf of X and Y. Therefore INDEPENDENCE, COVARIANCE AND CORRELATION M384G/374G Independence: For random variables and, the intuitive idea behind " is independent of " is that the distribution of shouldn't depend on what is. This

More information

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

More information

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i ) Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

More information

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

More information

POL 571: Expectation and Functions of Random Variables

POL 571: Expectation and Functions of Random Variables POL 571: Expectation and Functions of Random Variables Kosuke Imai Department of Politics, Princeton University March 10, 2006 1 Expectation and Independence To gain further insights about the behavior

More information

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions...

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions... MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo c Copyright 2004-2012 by Jenny A. Baglivo. All Rights Reserved. Contents 3 MT426 Notebook 3 3 3.1 Definitions............................................

More information

Probability and Statistical Methods. Chapter 4 Mathematical Expectation

Probability and Statistical Methods. Chapter 4 Mathematical Expectation Math 3 Chapter 4 Mathematical Epectation Mean of a Random Variable Definition. Let be a random variable with probability distribution f( ). The mean or epected value of is, f( ) µ = µ = E =, if is a discrete

More information

Some probability and statistics

Some probability and statistics Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,

More information

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

More information

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Module No. #01 Lecture No. #15 Special Distributions-VI Today, I am going to introduce

More information

FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL

FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint

More information

Review of Random Variables

Review of Random Variables Chapter 1 Review of Random Variables Updated: January 16, 2015 This chapter reviews basic probability concepts that are necessary for the modeling and statistical analysis of financial data. 1.1 Random

More information

E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

More information

Michigan State University Department of Electrical and Computer Engineering. ECE 280: ELECTRICAL ENGINEERING ANALYSIS (Spring 2008) Overview

Michigan State University Department of Electrical and Computer Engineering. ECE 280: ELECTRICAL ENGINEERING ANALYSIS (Spring 2008) Overview Michigan State University Department of Electrical and Computer Engineering ECE 280: ELECTRICAL ENGINEERING ANALYSIS (Spring 2008) Overview Staff Lectures Instructor (First Half of Course: 01/07/08 02/22/08)

More information

Digital Imaging and Multimedia. Filters. Ahmed Elgammal Dept. of Computer Science Rutgers University

Digital Imaging and Multimedia. Filters. Ahmed Elgammal Dept. of Computer Science Rutgers University Digital Imaging and Multimedia Filters Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines What are Filters Linear Filters Convolution operation Properties of Linear Filters Application

More information

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

More information

Probability and Statistics

Probability and Statistics CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2-0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

More information

Random Vectors and the Variance Covariance Matrix

Random Vectors and the Variance Covariance Matrix Random Vectors and the Variance Covariance Matrix Definition 1. A random vector X is a vector (X 1, X 2,..., X p ) of jointly distributed random variables. As is customary in linear algebra, we will write

More information

MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

More information

Lecture 6: Discrete & Continuous Probability and Random Variables

Lecture 6: Discrete & Continuous Probability and Random Variables Lecture 6: Discrete & Continuous Probability and Random Variables D. Alex Hughes Math Camp September 17, 2015 D. Alex Hughes (Math Camp) Lecture 6: Discrete & Continuous Probability and Random September

More information

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S.

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S. ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB Sohail A. Dianat Rochester Institute of Technology, New York, U.S.A. Eli S. Saber Rochester Institute of Technology, New York, U.S.A. (g) CRC Press Taylor

More information

CS395T Computational Statistics with Application to Bioinformatics

CS395T Computational Statistics with Application to Bioinformatics CS395T Computational Statistics with Application to Bioinformatics Prof. William H. Press Spring Term, 2010 The University of Texas at Austin Unit 6: Multivariate Normal Distributions and Chi Square The

More information

4. Joint Distributions of Two Random Variables

4. Joint Distributions of Two Random Variables 4. Joint Distributions of Two Random Variables 4.1 Joint Distributions of Two Discrete Random Variables Suppose the discrete random variables X and Y have supports S X and S Y, respectively. The joint

More information

Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy. Blue vs. Orange. Review Jeopardy Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

More information

( ) is proportional to ( 10 + x)!2. Calculate the

( ) is proportional to ( 10 + x)!2. Calculate the PRACTICE EXAMINATION NUMBER 6. An insurance company eamines its pool of auto insurance customers and gathers the following information: i) All customers insure at least one car. ii) 64 of the customers

More information

Probability for Estimation (review)

Probability for Estimation (review) Probability for Estimation (review) In general, we want to develop an estimator for systems of the form: x = f x, u + η(t); y = h x + ω(t); ggggg y, ffff x We will primarily focus on discrete time linear

More information

C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)}

C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)} C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)} 1. EES 800: Econometrics I Simple linear regression and correlation analysis. Specification and estimation of a regression model. Interpretation of regression

More information

Stochastic Processes and Advanced Mathematical Finance. Laws of Large Numbers

Stochastic Processes and Advanced Mathematical Finance. Laws of Large Numbers Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebraska-Lincoln Lincoln, NE 68588-0130 http://www.math.unl.edu Voice: 402-472-3731 Fax: 402-472-8466 Stochastic Processes and Advanced

More information

Advanced Quantitative Methods: Mathematics (p)review

Advanced Quantitative Methods: Mathematics (p)review Advanced Quantitative Methods: Mathematics (p)review University College Dublin 22 January 2013 1 Course outline 2 3 Outline Course outline 1 Course outline 2 3 Introduction Course outline Topic: advanced

More information

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

More information

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not. Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

More information

Introduction to Probability

Introduction to Probability Introduction to Probability EE 179, Lecture 15, Handout #24 Probability theory gives a mathematical characterization for experiments with random outcomes. coin toss life of lightbulb binary data sequence

More information

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

Math 431 An Introduction to Probability. Final Exam Solutions

Math 431 An Introduction to Probability. Final Exam Solutions Math 43 An Introduction to Probability Final Eam Solutions. A continuous random variable X has cdf a for 0, F () = for 0 <

More information

Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

More information

BNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I

BNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential

More information

Exam 2 Review. 3. How to tell if an equation is linear? An equation is linear if it can be written, through simplification, in the form.

Exam 2 Review. 3. How to tell if an equation is linear? An equation is linear if it can be written, through simplification, in the form. Exam 2 Review Chapter 1 Section1 Do You Know: 1. What does it mean to solve an equation? To solve an equation is to find the solution set, that is, to find the set of all elements in the domain of the

More information

Covariance and Correlation. Consider the joint probability distribution f XY (x, y).

Covariance and Correlation. Consider the joint probability distribution f XY (x, y). Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 2: Section 5-2 Covariance and Correlation Consider the joint probability distribution f XY (x, y). Is there a relationship between X and Y? If so, what kind?

More information

Linear Dependence Tests

Linear Dependence Tests Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

More information

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20 Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

More information

Estimation with Minimum Mean Square Error

Estimation with Minimum Mean Square Error C H A P T E R 8 Estimation with Minimum Mean Square Error INTRODUCTION A recurring theme in this text and in much of communication, control and signal processing is that of making systematic estimates,

More information

Picture Perfect: The Mathematics of JPEG Compression

Picture Perfect: The Mathematics of JPEG Compression Picture Perfect: The Mathematics of JPEG Compression May 19, 2011 1 2 3 in 2D Sampling and the DCT in 2D 2D Compression Images Outline A typical color image might be 600 by 800 pixels. Images Outline A

More information

Summary of Probability

Summary of Probability Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

More information

Understanding and Applying Kalman Filtering

Understanding and Applying Kalman Filtering Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding

More information

Welcome to Stochastic Processes 1. Welcome to Aalborg University No. 1 of 31

Welcome to Stochastic Processes 1. Welcome to Aalborg University No. 1 of 31 Welcome to Stochastic Processes 1 Welcome to Aalborg University No. 1 of 31 Welcome to Aalborg University No. 2 of 31 Course Plan Part 1: Probability concepts, random variables and random processes Lecturer:

More information

3. Continuous Random Variables

3. Continuous Random Variables 3. Continuous Random Variables A continuous random variable is one which can take any value in an interval (or union of intervals) The values that can be taken by such a variable cannot be listed. Such

More information

University of California, Berkeley, Statistics 134: Concepts of Probability

University of California, Berkeley, Statistics 134: Concepts of Probability University of California, Berkeley, Statistics 134: Concepts of Probability Michael Lugo, Spring 211 Exam 2 solutions 1. A fair twenty-sided die has its faces labeled 1, 2, 3,..., 2. The die is rolled

More information

Lecture 7 Cogsci 109. Thurs. Oct. 12, 2006

Lecture 7 Cogsci 109. Thurs. Oct. 12, 2006 Lecture 7 Cogsci 109 Thurs. Oct. 12, 2006 Announcements Homework 2 is posted. Office hours Midterm is coming up somewhere in the next couple of weeks Anything lectured on, presented in section, in the

More information

Orthogonal Diagonalization of Symmetric Matrices

Orthogonal Diagonalization of Symmetric Matrices MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

More information

A Introduction to Matrix Algebra and Principal Components Analysis

A Introduction to Matrix Algebra and Principal Components Analysis A Introduction to Matrix Algebra and Principal Components Analysis Multivariate Methods in Education ERSH 8350 Lecture #2 August 24, 2011 ERSH 8350: Lecture 2 Today s Class An introduction to matrix algebra

More information

Pattern Recognition. Probability Theory

Pattern Recognition. Probability Theory Pattern Recognition Probability Theory Probability Space 2 is a three-tuple with: the set of elementary events algebra probability measure -algebra over is a system of subsets, i.e. ( is the power set)

More information

5. Conditional Expected Value

5. Conditional Expected Value Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 5. Conditional Expected Value As usual, our starting point is a random experiment with probability measure P on a sample space Ω. Suppose that X is

More information

Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability

Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability Chapter 1 Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (1999. Other treatments of probability theory include Gallant (1997, Casella & Berger

More information

Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

More information

Chapter 3 RANDOM VARIATE GENERATION

Chapter 3 RANDOM VARIATE GENERATION Chapter 3 RANDOM VARIATE GENERATION In order to do a Monte Carlo simulation either by hand or by computer, techniques must be developed for generating values of random variables having known distributions.

More information

A Coefficient of Variation for Skewed and Heavy-Tailed Insurance Losses. Michael R. Powers[ 1 ] Temple University and Tsinghua University

A Coefficient of Variation for Skewed and Heavy-Tailed Insurance Losses. Michael R. Powers[ 1 ] Temple University and Tsinghua University A Coefficient of Variation for Skewed and Heavy-Tailed Insurance Losses Michael R. Powers[ ] Temple University and Tsinghua University Thomas Y. Powers Yale University [June 2009] Abstract We propose a

More information

STUDY GUIDE LINEAR ALGEBRA. David C. Lay University of Maryland College Park AND ITS APPLICATIONS THIRD EDITION UPDATE

STUDY GUIDE LINEAR ALGEBRA. David C. Lay University of Maryland College Park AND ITS APPLICATIONS THIRD EDITION UPDATE STUDY GUIDE LINEAR ALGEBRA AND ITS APPLICATIONS THIRD EDITION UPDATE David C. Lay University of Maryland College Park Copyright 2006 Pearson Addison-Wesley. All rights reserved. Reproduced by Pearson Addison-Wesley

More information

Multicollinearity in Regression Models

Multicollinearity in Regression Models 00 Jeeshim and KUCC65. (003-05-09) Multicollinearity.doc Introduction Multicollinearity in Regression Models Multicollinearity is a high degree of correlation (linear dependency) among several independent

More information

Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

More information

Introduction to Simulation Using MATLAB

Introduction to Simulation Using MATLAB Chapter 12 Introduction to Simulation Using MATLAB A. Rakhshan and H. Pishro-Nik 12.1 Analysis versus Computer Simulation A computer simulation is a computer program which attempts to represent the real

More information

Mean value theorem, Taylors Theorem, Maxima and Minima.

Mean value theorem, Taylors Theorem, Maxima and Minima. MA 001 Preparatory Mathematics I. Complex numbers as ordered pairs. Argand s diagram. Triangle inequality. De Moivre s Theorem. Algebra: Quadratic equations and express-ions. Permutations and Combinations.

More information

Continuous Random Variables

Continuous Random Variables Continuous Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Continuous Random Variables 2 11 Introduction 2 12 Probability Density Functions 3 13 Transformations 5 2 Mean, Variance and Quantiles

More information

Jointly Distributed Random Variables

Jointly Distributed Random Variables Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................

More information

Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..

Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,

More information

15.062 Data Mining: Algorithms and Applications Matrix Math Review

15.062 Data Mining: Algorithms and Applications Matrix Math Review .6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

More information

1 Notes on Markov Chains

1 Notes on Markov Chains Notes on Markov Chains A Markov chain is a finite-state process that can be characterized completely by two objects the finite set of states Z = {z,, z n } and the transition matrix Π = [π ij i,j=,n that

More information

Lesson 20. Probability and Cumulative Distribution Functions

Lesson 20. Probability and Cumulative Distribution Functions Lesson 20 Probability and Cumulative Distribution Functions Recall If p(x) is a density function for some characteristic of a population, then Recall If p(x) is a density function for some characteristic

More information

Introduction to Random Variables (RVs)

Introduction to Random Variables (RVs) ECE270: Handout 5 Introduction to Random Variables (RVs) Outline: 1. informal definition of a RV, 2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV, 3. a general rule to find probability

More information

ECE302 Spring 2006 HW5 Solutions February 21, 2006 1

ECE302 Spring 2006 HW5 Solutions February 21, 2006 1 ECE3 Spring 6 HW5 Solutions February 1, 6 1 Solutions to HW5 Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics

More information

Time Series Analysis

Time Series Analysis Time Series Analysis Time series and stochastic processes Andrés M. Alonso Carolina García-Martos Universidad Carlos III de Madrid Universidad Politécnica de Madrid June July, 2012 Alonso and García-Martos

More information

Analysis of Load Frequency Control Performance Assessment Criteria

Analysis of Load Frequency Control Performance Assessment Criteria 520 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 16, NO. 3, AUGUST 2001 Analysis of Load Frequency Control Performance Assessment Criteria George Gross, Fellow, IEEE and Jeong Woo Lee Abstract This paper presents

More information

1.5 Elementary Matrices and a Method for Finding the Inverse

1.5 Elementary Matrices and a Method for Finding the Inverse .5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

More information

Chapter 2, part 2. Petter Mostad

Chapter 2, part 2. Petter Mostad Chapter 2, part 2 Petter Mostad mostad@chalmers.se Parametrical families of probability distributions How can we solve the problem of learning about the population distribution from the sample? Usual procedure:

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Senior Secondary Australian Curriculum

Senior Secondary Australian Curriculum Senior Secondary Australian Curriculum Mathematical Methods Glossary Unit 1 Functions and graphs Asymptote A line is an asymptote to a curve if the distance between the line and the curve approaches zero

More information

Inner Product Spaces

Inner Product Spaces Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

More information

Mathematical Background

Mathematical Background Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,

More information

Lecture 7: Continuous Random Variables

Lecture 7: Continuous Random Variables Lecture 7: Continuous Random Variables 21 September 2005 1 Our First Continuous Random Variable The back of the lecture hall is roughly 10 meters across. Suppose it were exactly 10 meters, and consider

More information

Chapter 4 Expected Values

Chapter 4 Expected Values Chapter 4 Expected Values 4. The Expected Value of a Random Variables Definition. Let X be a random variable having a pdf f(x). Also, suppose the the following conditions are satisfied: x f(x) converges

More information

Generalized Linear Model Theory

Generalized Linear Model Theory Appendix B Generalized Linear Model Theory We describe the generalized linear model as formulated by Nelder and Wedderburn (1972), and discuss estimation of the parameters and tests of hypotheses. B.1

More information

Continuous Probability Distributions (120 Points)

Continuous Probability Distributions (120 Points) Economics : Statistics for Economics Problem Set 5 - Suggested Solutions University of Notre Dame Instructor: Julio Garín Spring 22 Continuous Probability Distributions 2 Points. A management consulting

More information

Probability and Statistics Vocabulary List (Definitions for Middle School Teachers)

Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence

More information

Machine Learning Math Essentials

Machine Learning Math Essentials Machine Learning Math Essentials Jeff Howbert Introduction to Machine Learning Winter 2012 1 Areas of math essential to machine learning Machine learning is part of both statistics and computer science

More information

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula derived from the Poisson Distribution

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula derived from the Poisson Distribution Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebrasa-Lincoln Lincoln, NE 68588-0130 http://www.math.unl.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics in Probability Theory

More information

Non Linear Dependence Structures: a Copula Opinion Approach in Portfolio Optimization

Non Linear Dependence Structures: a Copula Opinion Approach in Portfolio Optimization Non Linear Dependence Structures: a Copula Opinion Approach in Portfolio Optimization Jean- Damien Villiers ESSEC Business School Master of Sciences in Management Grande Ecole September 2013 1 Non Linear

More information

STAT/MTHE 353: Probability II. STAT/MTHE 353: Multiple Random Variables. Review. Administrative details. Instructor: TamasLinder

STAT/MTHE 353: Probability II. STAT/MTHE 353: Multiple Random Variables. Review. Administrative details. Instructor: TamasLinder STAT/MTHE 353: Probability II STAT/MTHE 353: Multiple Random Variables Administrative details Instructor: TamasLinder Email: linder@mast.queensu.ca T. Linder ueen s University Winter 2012 O ce: Je ery

More information

STAT 830 Convergence in Distribution

STAT 830 Convergence in Distribution STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2011 1 / 31

More information

CA200 Quantitative Analysis for Business Decisions. File name: CA200_Section_04A_StatisticsIntroduction

CA200 Quantitative Analysis for Business Decisions. File name: CA200_Section_04A_StatisticsIntroduction CA200 Quantitative Analysis for Business Decisions File name: CA200_Section_04A_StatisticsIntroduction Table of Contents 4. Introduction to Statistics... 1 4.1 Overview... 3 4.2 Discrete or continuous

More information