Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 11 04/01/2008. Sven Zenker
|
|
- William Parker
- 7 years ago
- Views:
Transcription
1 Parameter estimatio for oliear models: Numerical approaches to solvig the iverse problem Lecture 11 04/01/2008 Sve Zeker
2 Review: Trasformatio of radom variables Cosider probability distributio of a radom variable X described by PDF f ( x) defied o. How ca we fid the PDF f ( y) of a ew radom variable X we arrive at by applyig a ivertible fuctio T :, y = T( x) to the origial oe? Cosider probability of y beig i some subset S of Py ( S) f( ydy ), which we could compute if we kew f Y. = S Y Sice T is ivertible, we ca express the above itegral i terms of f ( x) as follows: Py ( S) = f( ydy ) = f ( xdx ) Y S 1 T ( S) ad chage variables to y to obtai 1 1 ( ) Y( ) X( ) X( ( )) det y ( ) S 1 T ( S) S so we see that 1 1 Y( ) X( ( )) det y ( ) X Py S = f ydy= f xdx= f T y DT ydy f y = f T y D T y Y X
3 Review: Expected value For a radom variable X described by a probability desity fuctio f EX ( ) = xf( xdx ) X ( x), the expected value is The expected value of a fuctio gx ( ) of the radom variable is EgX ( ( )) = gx ( ) f( xdx )
4 Review: Bayesia iferece for cotious variables f XY ( x y) = f ( y x) f ( x) Y X X f ( y x) f ( x) dx Probability desity fuctio of the posterior distributio
5 Review: Prior distributios Prior distributios have to be kow or assumed to be able to perform Bayesia iferece I the Bayesia spirit, they ca be used ecode prior beliefs about parameter/state values Approaches to objectifyig prior selectio by formally describig a state of miimal iformatio, typically based o ivariace argumets or iformatio theoretic ideas, exist (Jeffrey s prior, referece priors) I practice, if sufficiet data are available, the effect of prior selectio may be small This ca be explored experimetally
6 Samplig to tackle high dimesioal problems A way out: sample based approximatio... If we ca obtai a set of samples { X,..., X } from the posterior distributio π ( x), 1 E( f( x)) = f( x) π ( x) dx f( Xi ) i= 1 1
7 Review homework Key observatios: Product of margials equal to joit PDF iff radom variables are idepedet Sample based approximatio of margial PDFs works ad gets better with larger sample size Histogram ormalizatio factor: c bi = w bi bi N
8 From sample to PDF: Desity estimatio Histogram a istace of more geeral problem of desity estimatio: Give a fiite set of samples from a probability distributio, how ca I approximately fid the PDF of the uderlyig distributio? Histogram oe (relatively crude) way
9 Histogram: bi width selectio Various formulas based o asymptotic argumets exist. All of them (of course) have bi width decrease with icreasig umber of samples, ad also take ito accout some measure of the spread of the data... E.g.: 7s w = (Scott's rule) where umber of samples, s stadard deviatio of sample bi or w bi = IQR (Freedma-Diacois rule), where IQR is the iterquartile rage of the data These formulas give oe bi width for the etire dataset. A variety of methods exist to further refie the bi widths by allowig them to chage locally
10 Aother approach: Kerel desity estimatio Differet idea: rather tha coutig samples i a give rage, a probability desity bump is placed at the locatio of each sample (the kerel), the idea beig that the presece of a sample supports a higher desity aroud the positio of that sample. These kerels are all added up ad ormalized (source: Wikimedia commos,
11 Kerel desity estimatio (KDE) For a i.i.d. sample { x,..., x of the uderlyig distributio is give by 1 N }, the (fixed badwidth) kerel desity estimate for the PDF N 1 x x ˆ( ) ( i f x = K ), Nh h i= 1 where K( x) is a kerel fuctio satisfyig K( xdx ) = 1 Usually, oe will also require K( x) 0 ad K( x) = K(- x) for all x h is termed the badwith. Its selectio is critical for performace of the estimator ad much more importat tha the specific shape of the kerel used.
12 KDE: Effect of badwidth Source: Wikimedia Commos,
13 KDE: Adaptive badwidth selectio Badwidth ca also be chose for each idividual sample separately, leadig to much improved estimates (lower desity -> wider kerels, higher desity -> > arrower kerels) The optimal badwidth selectio problem i higher dimesios is still a active area of research Importat idea: pealized MLE approaches (Why? Dirac catastrophe )
14 From PDF to sample: the samplig problem Wish to create set of i.i.d.. samples give a PDF kow up to a costat scalar multiple, e.g. to solve f XY ( x y) = f ( y x) f ( x) YX X f( y x) f( x) dx Deomiator (ormalizatio costat) usually ot kow
15 Markov Chai Mote Carlo While ideas like samplig from uiform distributio o [0,1] ad the trasformig with the iverse CDF work for simple PDFs kow i closed form, they are hopeless for realistic scearios The techiques we will discuss will allow us to obtai samples that, for a large umber of samples, are distributed accordig to the PDF we provide (our Bayesia posterior, e.g.) The idividual samples will ot be idepedet
16 Markov Chais for cotiuous state spaces Idea: defie a discrete time stochastic process o a cotious state space, that is, a descriptio of a particle movig aroud i our cotiuous state space i a radom fashio, e.g., i the followig way Trasitio kerel Px (, A) with x, A Σ, Σ Borel sigma field over (i.e., A is a subset of that is "ice eough") gives the probability that our stochastic process moves from x to a poit i the set A. P is a coditioal distributio fuctio givig the probability of movig to a certai subset of, give that we are at x. I particular, Px (, ) = 1, that is, we are certai to move somwhere... However, Px (,{ x}) eed ot be zero, that is, our chai ca stay where it is...
17 Ivariat distributios A large body of theory exists that treats 2 key questios arisig whe cosiderig the log term behavior of such Markov chais i cotiuous state spaces: * a) existece of ivariat distributio with *, that is ( ) ( ) satisfyig π π dy = π y dy * π ( dy) = P( x, dy) π( x) dx π desity wrt. Lebesgue measure that is, the ivariat distributio does ot chage whe the trasitio kerel acts o it... b) coditios uder which a iteratio of the trasitio kerel give by (, ) (, ) (, ) ( ) ( 1) P x A = P x dy P y A (1) ad (, ) (, ) P x dy = P x dy * coverges to the statioary distributio π
18 Markov Chai Mote Carlo Idea: Create a trasitio kerel that has the distributio we wish to sample from as ivariat distributio ad coverges to it Seems hard, but ca i fact be doe
19 Markov Chai Mote Carlo Suppose trasitio kerel takes form Pxdy (, ) = pxydy (, ) + rx ( ) δ ( dy) x where pxx (, ) = 0 ad δ ( dy) = 1 if x i dy ad 0 otherwise ad x rx ( ) 1 pxydy (, ) is the probability that = the chai remais at x.
20 Markov Chai Mote Carlo (MCMC) Now, if pxy (, ) satisfies the so-callled "detailed balace" coditio π( x) p( x, y) = π( y) p( y, x) the π ( )is the ivariat desity of Px (, ).
21 MCMC Evaluate RHS of statioarity coditio, usig assumed form of trasitio kerel ad liearity of itegratio (lie 1), switchig the order of itegratio (lie 2), ad usig the assumed detailed balace coditio (lie 3), to see that if p(x,y) i the proposed trasitio kerel satisfies detailed balace, \pi is ideed a statioary distributio of the Markov chai... From Chib et al., The America Statisticia 1995, Vol 49(4),
22 How to arrive at a trasitio kerel that satisfies detailed balace We ca choose some arbitrary family of probability desities qxy (, ) that ca deped oly o the curret state (Markov property!). These will i geeral ot satisfy the detailed balace criterio eeded for the chai to have the desired target distributio π which we are aimig to sample from (e.g., our Bayesia posterior). For example, it could be that π( xqxy ) (, ) > π( yqyx ) (, ) So, roughly speakig, we move from x to y too ofte, ad ot frequetly eough from y to x. A coveiet way to compesate for this, which leads to Metropolis-Hastigs algorithms, is to itroduce a probability α ( xy, ) < 1 that the move from x to y is actually made. If the move is ot made, the process stays at x. So the the specific form for pxy (, ) i the Metropolis-Hastigs algorithm becomes pmh ( x, y) = q( x, y) α( x, y), x y For the case where the iequality is as above, we may wish to set α( yx, ) = 1, the largest possible value for a probability, ad ca the compute α( xy, ) from the detailed balace coditio π( xqxy ) (, ) α( xy, ) = π( yqyx ) (, ) α( y, x) to obtai π ( yqyx ) (, ) α( xy, ) =, π ( xqxy ) (, ) ad similarly for the iequality i the other directio by settig α( xy, ) = 1.
23 How to arrive at a trasitio kerel that satisfies detailed balace So to obtai detailed balace/reversibility, we chose π ( yqyx ) (, ) mi,1 if π ( xqxy ) (, ) > 0 α( xy, ) = π ( xqxy ) (, ) 1 otherwise which, together with the probability for stayig at the curret positio rx ( ) = 1 qxy (, ) α( xydy, ) yields a overall trasitio kerel which is a special case of the versio from the previous slide: P MH ( xdy, ) = qxy (, ) α( xydy, ) + 1 qxy (, ) α( xydy, ) δx( dy) for which we saw that it does have the desired ivariat distributio sice we have detailed balace/reversibility by costructio...
24 Observatios -For a symmetric proposal distributio, that is pxy (, ) = pyx (, ), π ( y) α( xy, ) =, so that "uphill moves" will always be accepted π ( x) (simulated aealig!) -For qxy (, ) = π( y), α( xy, ) = 1, so if the proposal distributio is the true distributio we wish to sample from, we will always accept the move -The PDF of the distributio of iterest π eed oly be kow up to a costat scalar factor sice it appears both i umerator ad deomiator
25 Algorithm Iitializatio Specify Family of proposal distributios q(x,y) Desired umber of samples N Iitial value x 0 Mai loop Repeat for j=1, N Geerate (sample) y from q(x j,.) ad u from uiform distributio o [0,1] If u <= α(x j,y) Set x j+1 = y Else Set x j+1 =x j Ed Repeat Termiatio Retur the set of samples {x 1,,x N }
26 Assigmet o. 9 Implemet the described Metropolis-Hastigs samplig algorithm for a geeric family of symmetric proposal distributio q(x,y) ) ad target desity i MATLAB the fuctio header could for example look like this: fuctio [samples, accept] ] = mh_symm(samples,, x0, sampleq,, des) % mh_symm implemets a simple Metropolis-Hastigs sampler for the PDF des usig the % symmetric proposal distributio from which sampleq samples % samples - umber of iteratios/correlated samples to be draw % x0 - colum vector givig iitial positio i parameter space % sampleq - fuctio sampleq(x) ) should retur sample from proposal % distributio for curret positio x % des - fuctio des(x) ) should retur desity of PDF correspodig % to distributio we wish to sample % returs % samples - legth(x0) x iter array of samples % accept - umber of accepted steps Explore its behavior by utilizig the 2-dimesioal 2 test distributios provided for homework o. 7 as target distributios ad ruig the sampler usig symmetric bivariate ormal distributios with idetical margial stadard deviatios \sigma as proposal distributio (you ca sample from these by simply samplig from the margials usig rad,, thaks to idepedece ). Ru the sampler for iteratios for proposal sigma = 0.01, 0.1, 1, 10, 15, 20, ad 40, respectively, usig [0;0] for x0 every time for comparability for both distributios Compute the acceptace ratio, that is (o. of accepted steps)/(total umber of iteratios), for each value of sigma. What do you observe? Use your (corrected, if ecessary) ormalized histogram code from m homework o. 7 to compare the estimated margial desities you obtai from each sample with the e true margial desities. What do you observe for the differet choices of sigma for the 2 distributios? How do you iterpret your overall fidigs?
Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 12 04/08/2008. Sven Zenker
Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem Lecture 12 04/08/2008 Sven Zenker Assignment no. 8 Correct setup of likelihood function One fixed set of observation
More informationProperties of MLE: consistency, asymptotic normality. Fisher information.
Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout
More informationChapter 6: Variance, the law of large numbers and the Monte-Carlo method
Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value
More informationCase Study. Normal and t Distributions. Density Plot. Normal Distributions
Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca
More informationChapter 7 Methods of Finding Estimators
Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of
More informationI. Chi-squared Distributions
1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.
More informationUniversity of California, Los Angeles Department of Statistics. Distributions related to the normal distribution
Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More informationOverview of some probability distributions.
Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability
More informationMaximum Likelihood Estimators.
Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio
More informationNormal Distribution.
Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued
More informationModified Line Search Method for Global Optimization
Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o
More informationLECTURE 13: Cross-validation
LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M
More information5: Introduction to Estimation
5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample
More informationConfidence Intervals for One Mean
Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a
More informationOutput Analysis (2, Chapters 10 &11 Law)
B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should
More informationThe following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles
The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio
More informationGCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.
GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all
More informationDepartment of Computer Science, University of Otago
Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly
More informationZ-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown
Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about
More informationChapter 14 Nonparametric Statistics
Chapter 14 Noparametric Statistics A.K.A. distributio-free statistics! Does ot deped o the populatio fittig ay particular type of distributio (e.g, ormal). Sice these methods make fewer assumptios, they
More informationCHAPTER 7: Central Limit Theorem: CLT for Averages (Means)
CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:
More informationCenter, Spread, and Shape in Inference: Claims, Caveats, and Insights
Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the
More information1. C. The formula for the confidence interval for a population mean is: x t, which was
s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value
More informationExploratory Data Analysis
1 Exploratory Data Aalysis Exploratory data aalysis is ofte the rst step i a statistical aalysis, for it helps uderstadig the mai features of the particular sample that a aalyst is usig. Itelliget descriptios
More informationPlug-in martingales for testing exchangeability on-line
Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk
More informationBASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)
BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet
More informationHypothesis testing. Null and alternative hypotheses
Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate
More informationWeek 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable
Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5
More informationConvexity, Inequalities, and Norms
Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for
More information1 Computing the Standard Deviation of Sample Means
Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.
More informationA probabilistic proof of a binomial identity
A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two
More informationStatistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
More informationSoving Recurrence Relations
Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree
More informationIn nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008
I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces
More informationIncremental calculation of weighted mean and variance
Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically
More informationSolutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork
Solutios to Selected Problems I: Patter Classificatio by Duda, Hart, Stork Joh L. Weatherwax February 4, 008 Problem Solutios Chapter Bayesia Decisio Theory Problem radomized rules Part a: Let Rx be the
More informationDiscrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may
More informationMeasures of Spread and Boxplots Discrete Math, Section 9.4
Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,
More informationLecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)
18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the
More informationChapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:
Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries
More informationSampling Distribution And Central Limit Theorem
() Samplig Distributio & Cetral Limit Samplig Distributio Ad Cetral Limit Samplig distributio of the sample mea If we sample a umber of samples (say k samples where k is very large umber) each of size,
More informationInference on Proportion. Chapter 8 Tests of Statistical Hypotheses. Sampling Distribution of Sample Proportion. Confidence Interval
Chapter 8 Tests of Statistical Hypotheses 8. Tests about Proportios HT - Iferece o Proportio Parameter: Populatio Proportio p (or π) (Percetage of people has o health isurace) x Statistic: Sample Proportio
More informationVladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT
Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee
More informationarxiv:1506.03481v1 [stat.me] 10 Jun 2015
BEHAVIOUR OF ABC FOR BIG DATA By Wetao Li ad Paul Fearhead Lacaster Uiversity arxiv:1506.03481v1 [stat.me] 10 Ju 2015 May statistical applicatios ivolve models that it is difficult to evaluate the likelihood,
More informationDetermining the sample size
Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors
More informationIrreducible polynomials with consecutive zero coefficients
Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem
More informationSAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx
SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval
More informationSequences and Series
CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their
More informationConfidence Intervals. CI for a population mean (σ is known and n > 30 or the variable is normally distributed in the.
Cofidece Itervals A cofidece iterval is a iterval whose purpose is to estimate a parameter (a umber that could, i theory, be calculated from the populatio, if measuremets were available for the whole populatio).
More informationThe Stable Marriage Problem
The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,
More informationLesson 17 Pearson s Correlation Coefficient
Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig
More informationRunning Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis
Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.
More informationHypergeometric Distributions
7.4 Hypergeometric Distributios Whe choosig the startig lie-up for a game, a coach obviously has to choose a differet player for each positio. Similarly, whe a uio elects delegates for a covetio or you
More informationTHE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n
We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 8
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 8 GENE H GOLUB 1 Positive Defiite Matrices A matrix A is positive defiite if x Ax > 0 for all ozero x A positive defiite matrix has real ad positive
More information.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth
Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,
More informationCOMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S 2 CONTROL CHART FOR THE CHANGES IN A PROCESS
COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S CONTROL CHART FOR THE CHANGES IN A PROCESS Supraee Lisawadi Departmet of Mathematics ad Statistics, Faculty of Sciece ad Techoology, Thammasat
More informationSECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES
SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,
More information, a Wishart distribution with n -1 degrees of freedom and scale matrix.
UMEÅ UNIVERSITET Matematisk-statistiska istitutioe Multivariat dataaalys D MSTD79 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multivariat dataaalys D, 5 poäg.. Assume that
More information1 The Gaussian channel
ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.
More informationQuadrat Sampling in Population Ecology
Quadrat Samplig i Populatio Ecology Backgroud Estimatig the abudace of orgaisms. Ecology is ofte referred to as the "study of distributio ad abudace". This beig true, we would ofte like to kow how may
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006
Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam
More informationRainbow options. A rainbow is an option on a basket that pays in its most common form, a nonequally
Raibow optios INRODUCION A raibow is a optio o a basket that pays i its most commo form, a oequally weighted average of the assets of the basket accordig to their performace. he umber of assets is called
More informationDescriptive Statistics
Descriptive Statistics We leared to describe data sets graphically. We ca also describe a data set umerically. Measures of Locatio Defiitio The sample mea is the arithmetic average of values. We deote
More informationPROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM
PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics
More informationMEI Structured Mathematics. Module Summary Sheets. Statistics 2 (Version B: reference to new book)
MEI Mathematics i Educatio ad Idustry MEI Structured Mathematics Module Summary Sheets Statistics (Versio B: referece to ew book) Topic : The Poisso Distributio Topic : The Normal Distributio Topic 3:
More informationSection 11.3: The Integral Test
Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult
More informationMath C067 Sampling Distributions
Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters
More informationhp calculators HP 12C Statistics - average and standard deviation Average and standard deviation concepts HP12C average and standard deviation
HP 1C Statistics - average ad stadard deviatio Average ad stadard deviatio cocepts HP1C average ad stadard deviatio Practice calculatig averages ad stadard deviatios with oe or two variables HP 1C Statistics
More informationUniversal coding for classes of sources
Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric
More informationOverview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals
Overview Estimatig the Value of a Parameter Usig Cofidece Itervals We apply the results about the sample mea the problem of estimatio Estimatio is the process of usig sample data estimate the value of
More informationCS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations
CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad
More informationOne-sample test of proportions
Oe-sample test of proportios The Settig: Idividuals i some populatio ca be classified ito oe of two categories. You wat to make iferece about the proportio i each category, so you draw a sample. Examples:
More informationLecture 4: Cheeger s Inequality
Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular
More informationPartial Di erential Equations
Partial Di eretial Equatios Partial Di eretial Equatios Much of moder sciece, egieerig, ad mathematics is based o the study of partial di eretial equatios, where a partial di eretial equatio is a equatio
More informationSystems Design Project: Indoor Location of Wireless Devices
Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 698-5295 Email: bcm1@cec.wustl.edu Supervised
More informationThis document contains a collection of formulas and constants useful for SPC chart construction. It assumes you are already familiar with SPC.
SPC Formulas ad Tables 1 This documet cotais a collectio of formulas ad costats useful for SPC chart costructio. It assumes you are already familiar with SPC. Termiology Geerally, a bar draw over a symbol
More informationChapter 7: Confidence Interval and Sample Size
Chapter 7: Cofidece Iterval ad Sample Size Learig Objectives Upo successful completio of Chapter 7, you will be able to: Fid the cofidece iterval for the mea, proportio, ad variace. Determie the miimum
More informationAsymptotic Growth of Functions
CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll
More informationLecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k.
18.409 A Algorithmist s Toolkit September 17, 009 Lecture 3 Lecturer: Joatha Keler Scribe: Adre Wibisoo 1 Outlie Today s lecture covers three mai parts: Courat-Fischer formula ad Rayleigh quotiets The
More informationINVESTMENT PERFORMANCE COUNCIL (IPC)
INVESTMENT PEFOMANCE COUNCIL (IPC) INVITATION TO COMMENT: Global Ivestmet Performace Stadards (GIPS ) Guidace Statemet o Calculatio Methodology The Associatio for Ivestmet Maagemet ad esearch (AIM) seeks
More informationParametric (theoretical) probability distributions. (Wilks, Ch. 4) Discrete distributions: (e.g., yes/no; above normal, normal, below normal)
6 Parametric (theoretical) probability distributios. (Wilks, Ch. 4) Note: parametric: assume a theoretical distributio (e.g., Gauss) No-parametric: o assumptio made about the distributio Advatages of assumig
More informationCHAPTER 3 THE TIME VALUE OF MONEY
CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all
More informationHow To Calculate A Radom Umber From A Probability Fuctio
Iteral Report SUF PFY/96 Stockholm, December 996 st revisio, 3 October 998 last modificatio September 7 Had-book o STATISTICAL DISTRIBUTIONS for experimetalists by Christia Walck Particle Physics Group
More informationA Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length
Joural o Satisfiability, Boolea Modelig ad Computatio 1 2005) 49-60 A Faster Clause-Shorteig Algorithm for SAT with No Restrictio o Clause Legth Evgey Datsi Alexader Wolpert Departmet of Computer Sciece
More informationEkkehart Schlicht: Economic Surplus and Derived Demand
Ekkehart Schlicht: Ecoomic Surplus ad Derived Demad Muich Discussio Paper No. 2006-17 Departmet of Ecoomics Uiversity of Muich Volkswirtschaftliche Fakultät Ludwig-Maximilias-Uiversität Müche Olie at http://epub.ub.ui-mueche.de/940/
More informationLecture 5: Span, linear independence, bases, and dimension
Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;
More informationResearch Method (I) --Knowledge on Sampling (Simple Random Sampling)
Research Method (I) --Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact
More informationCS103X: Discrete Structures Homework 4 Solutions
CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible six-figure salaries i whole dollar amouts are there that cotai at least
More informationAn Efficient Polynomial Approximation of the Normal Distribution Function & Its Inverse Function
A Efficiet Polyomial Approximatio of the Normal Distributio Fuctio & Its Iverse Fuctio Wisto A. Richards, 1 Robi Atoie, * 1 Asho Sahai, ad 3 M. Raghuadh Acharya 1 Departmet of Mathematics & Computer Sciece;
More informationwhere: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return
EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The
More informationNOTES ON PROBABILITY Greg Lawler Last Updated: March 21, 2016
NOTES ON PROBBILITY Greg Lawler Last Updated: March 21, 2016 Overview This is a itroductio to the mathematical foudatios of probability theory. It is iteded as a supplemet or follow-up to a graduate course
More informationLecture 2: Karger s Min Cut Algorithm
priceto uiv. F 3 cos 5: Advaced Algorithm Desig Lecture : Karger s Mi Cut Algorithm Lecturer: Sajeev Arora Scribe:Sajeev Today s topic is simple but gorgeous: Karger s mi cut algorithm ad its extesio.
More informationFIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. 1. Powers of a matrix
FIBONACCI NUMBERS: AN APPLICATION OF LINEAR ALGEBRA. Powers of a matrix We begi with a propositio which illustrates the usefuless of the diagoalizatio. Recall that a square matrix A is diogaalizable if
More informationMARTINGALES AND A BASIC APPLICATION
MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this
More informationUncertainty Chapter 13. Mausam (Based on slides by UW-AI faculty)
Ucertait Chapter 3 Mausam Based o slides b UW-AI facult Kowledge Represetatio KR Laguage Otological Commitmet Epistemological Commitmet ropositioal Logic facts true false ukow First Order Logic facts objects
More information10-705/36-705 Intermediate Statistics
0-705/36-705 Itermediate Statistics Larry Wasserma http://www.stat.cmu.edu/~larry/=stat705/ Fall 0 Week Class I Class II Day III Class IV Syllabus August 9 Review Review, Iequalities Iequalities September
More informationINFINITE SERIES KEITH CONRAD
INFINITE SERIES KEITH CONRAD. Itroductio The two basic cocepts of calculus, differetiatio ad itegratio, are defied i terms of limits (Newto quotiets ad Riema sums). I additio to these is a third fudametal
More informationEntropy of bi-capacities
Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics
More information