# The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs

Save this PDF as:

Size: px
Start display at page:

## Transcription

4 LIU, LAFFERTY, AND WASSERMAN where Σ 0 is the covariace of X uder. IfS is a set of covariace matrices, the oracle is defied to be the covariace matrix Σ that miimizes RΣ overs: Σ = arg mi Σ S RΣ. Thus px;µ 0,Σ is the best predictor of a ew observatio amog all distributios i {px;µ 0,Σ : Σ S}. I particular, ifs cosists of covariace matrices with sparse graphs, the px;µ 0,Σ is, i some sese, the best sparse predictor. A estimator Σ is persistet if R Σ RΣ 0 as the sample size icreases to ifiity. Thus, a persistet estimator approximates the best estimator over the classs, but we do ot assume that the true distributio has a covariace matrix is, or eve that it is Gaussia. Moreover, we allow the dimesio p = p to icrease with. O the other had, orm cosistecy ad sparsistecy require that the true distributio is Gaussia. I this case, let Σ 0 deote the true covariace matrix. A estimator is orm cosistet if Σ Σ 0 where is a orm. If EΩ deotes the edge set correspodig to Ω, a estimator is sparsistet if EΩ E Ω 0. Thus, a sparsistet estimator idetifies the correct graph cosistetly. We preset our theoretical aalysis o these properties of the oparaormal i Sectio The Noparaormal We say that a radom vector X = X,...,X p T has a oparaormal distributio if there exist fuctios { f j } p j= such that Z fx Nµ,Σ, where fx = f X,..., f p X p. We the write X NN µ,σ, f. Whe the f j s are mootoe ad differetiable, the joit probability desity fuctio of X is give by { p X x = 2π p/2 exp p Σ /2 2 fx µt Σ fx µ} f jx j. 2 j= Lemma The oparaormal distributio NN µ,σ, f is a Gaussia copula whe the f j s are mootoe ad differetiable. roof By Sklar s theorem Sklar, 959, ay joit distributio ca be writte as Fx,...,x p = C{F x,...,f p x p } where the fuctio C is called a copula. For the oparaormal we have Fx,...,x p = Φ µ,σ Φ F x,...,φ F p x p 2298

6 LIU, LAFFERTY, AND WASSERMAN Figure 2: Desities of three 2-dimesioal oparaormals. The compoet fuctios have the form f j x = sigx x α j. Left: α = 0.9, α 2 = 0.8; ceter: α =.2, α 2 = 0.8; right α = 2, α 2 = 3. I each case µ= 0,0 ad Σ =.5.5. Thus, it is ot ecessary to estimate µ or σ to estimate the graph. Figure 2 shows three examples of 2-dimesioal oparaormal desities. I each case, the compoet fuctios f j x take the form f j x = a j sigx x α j + b j where the costats a j ad b j are set to eforce the idetifiability costraits 3. The covariace i each case is Σ =.5.5 ad the mea is µ= 0,0. The expoet α j determies the oliearity. It ca be see how the cocavity of the desity chages with the expoet α, ad that α > ca result i multiple modes. The assumptio that fx = f X,..., f p X p is ormal leads to a semiparametric model where oly oe dimesioal fuctios eed to be estimated. But the mootoicity of the fuctios f j, which map otor, eables computatioal tractability of the oparaormal. For more geeral fuctios f, the ormalizig costat for the desity { p X x exp } 2 fx µt Σ fx µ caot be computed i closed form. 2300

9 THE NONARANORMAL Theorem 4 Suppose that p = ξ ad let f be the Wisorized estimator defied i 7 with δ = 4 /4 πlog. Defie For some M 2ξ+. The for ay ε C M log plog 2 C M 48 π 2M M /2 ad sufficietly large, we have max S f jk S f jk > 2ε jk 2 πlogp + 2exp /2 ε 2 2log p 232π 2 log 2 + 2exp 2log p /2 + o. 8πlog The proof of the above theorem is give i Sectio 7. The followig corollary is immediate, ad specifies the scalig of the dimesio i terms of sample size. Corollary 5 Let M max{5π,2ξ+}. The log plog max 2 S f jk S f jk > 2CM jk /2 = o. Hece, max j,k S f jk S f jk = O log plog 2. /2 The followig corollary yields estimatio cosistecy i both the Frobeius orm ad the l 2 - operator orm. The proof follows the same argumets as the proof of Theorem ad Theorem 2 from Rothma et al. 2008, replacig their Lemma with our Theorem 4. For a matrix A = a i j, the Frobeius orm F is defied as A F i, j a 2 i j. The l 2- operator orm 2 is defied as the magitude of the largest eigevalue of the matrix, A 2 max x 2 = Ax 2. I the followig, we write a b if there are positive costats c ad C idepedet of such that c a /b C. Corollary 6 Suppose that the data are geerated as X i NN µ 0,Σ 0, f 0, ad let Ω 0 = Σ 0. If the regularizatio parameter λ is chose as log plog 2 λ 2C M /2 where C M is defied i Theorem 4. The the oparaormal estimator Ω of 9 satisfies Ω Ω 0 F = O s+ plog plog 2 /2 2303

11 THE NONARANORMAL Our persistecy risk cosistecy result parallels the persistecy result for additive models give i Ravikumar et al. 2009a, ad allows model dimesio that grows expoetially with sample size. The defiitio i this theorem uses the fact from Lemma that sup x Φ F j x 2log whe δ = /4 /4 πlog. I the ext theorem, we do ot assume the true model is oparaormal ad defie the populatio ad sample risks as R f,ω = 2 { tr [ ΩE fx fx T ] log Ω plog2π } R f,ω = 2 {tr[ωs f] log Ω plog2π}. Theorem 8 Suppose that p e ξ for some ξ <, ad defie the classes M = { f :R R : f is mootoe with f C } log C = { Ω : Ω L }. Let Ω be give by The R f, Ω { } Ω = argmi tr ΩS f log Ω. Ω C log if R f,ω = O L C ξ. f,ω M p Hece the Wisorized estimator of f,ω with δ = /4 /4 πlog is persistet over C whe L = o ξ/2 / log. The proofs of Theorems 4 ad 8 are give i Sectio Experimetal Results I this sectio, we report experimetal results o sythetic ad real data sets. We maily compare the l -regularized oparaormal ad Gaussia paraormal models, computed usig the graphical lasso algorithm glasso of Friedma et al The primary coclusios are: i Whe the data are multivariate Gaussia, the performace of the two methods is comparable; ii whe the model is correct, the oparaormal performs much better tha the graphical lasso i may cases; iii for a particular gee microarray data set, our method behaves differetly from the graphical lasso, ad may support differet biological coclusios. Note that we ca reuse the glasso implemetatio to fit a sparse oparaormal. I particular, after computig the Wisorized sample covariace S f, we pass this matrix to the glasso routie to carry out the optimizatio { } Ω = arg mi tr ΩS f log Ω +λ Ω. Ω 2305

12 LIU, LAFFERTY, AND WASSERMAN 6. Neighborhood Graphs We begi by describig a procedure to geerate graphs as i Meishause ad Bühlma, 2006, with respect to which several distributios ca the be defied. We geerate a p-dimesioal sparse graph G V,E as follows: Let V = {,..., p} correspod to variables X = X,...,X p. We associate each idex j with a poit Y [0,] 2 where j,y 2 j Y k,...,y k Uiform[0, ] for k =,2. Each pair of odes i, j is icluded i the edge set E with probability i, j E = exp y i y j 2 2π 2s where y i y i,y 2 i is the observatio of Y i,y 2 i ad represets the Euclidea distace. Here, s = 0.25 is a parameter that cotrols the sparsity level of the geerated graph. We restrict the maximum degree of the graph to be four ad build the iverse covariace matrix Ω 0 accordig to Ω 0 i, j = if i = j if i, j E 0 otherwise, where the value guaratees positive defiiteess of the iverse covariace matrix. Give Ω 0, data poits are sampled from X,...,X NNµ 0,Σ 0, f 0 where µ 0 =.5,...,.5, Σ 0 = Ω 0. For simplicity, the trasformatio fuctios for all dimesios are the same, f =...= f p = f. To sample data from the oparaormal distributio, we also require g f ; two differet trasformatios g are employed. Defiitio 9 Gaussia CDF Trasformatio Let g 0 be a oe-dimesioal Gaussia cumulative distributio fuctio with mea µ g0 ad the stadard deviatio σ g0, that is, t µg0 g 0 t Φ We defie the trasformatio fuctio g j = f j g j z j σ j where σ j = Σ 0 j, j. σ g0. for the j-th dimesio as Z t µj g 0 z j g 0 tφ σ j dt Z Z dt 2 t µj y µj g 0 y g 0 tφ σ j φ σ j dy + µ j 2306

14 LIU, LAFFERTY, AND WASSERMAN cdf power liear glasso path glasso path glasso path oparaormal path oparaormal path oparaormal path = 500 cdf power liear glasso path glasso path glasso path oparaormal path oparaormal path oparaormal path = 200 Figure 4: Regularizatio paths for the glasso ad oparaormal with = 500 top ad = 200 bottom. The paths for the relevat variables ozero iverse covariace etries are plotted as solid black lies; the paths for the irrelevat variables are plotted as dashed red lies. For o-gaussia distributios, the oparaormal better separates the relevat ad irrelevat dimesios. To geerate sythetic data, we set p = 40, resultig i = 820 parameters to be estimated, ad vary the sample sizes from = 200 to = 000. Three coditios are cosidered, correspodig to usig the cdf trasform, the power trasform, or o trasformatio. I each case, both the glasso ad the oparaormal are applied to estimate the graph. 2308

15 THE NONARANORMAL 6.. COMARISON OF REGULARIZATION ATHS We choose a set of regularizatio parameters Λ; for each λ Λ, we obtai a estimate Ω which is a matrix. The upper triagular matrix has 780 parameters; we vectorize it to get a 780-dimesioal parameter vector. A regularizatio path is the trace of these parameters over all the regularizatio parameters withi Λ. The regularizatio paths for both methods are plotted i Figure 4. For the cdf trasformatio ad the power trasformatio, the oparaormal separates the relevat ad the irrelevat dimesios very well. For the glasso, relevat variables are mixed with irrelevat variables. If o trasformatio is applied, the paths for both methods are almost the same ESTIMATED TRANSFORMATIONS For sample size = 000, we plot the estimated trasformatios for three of the variables i Figure 5. It is clear that Wisorizatio plays a sigificat role for the power trasformatio. This is ituitive due to the high skewess of the oparaormal distributio i this case. cdf power liear f estimated true f estimated true g estimated true x x x f estimated true f estimated true g estimated true x2 x2 x2 f estimated true f estimated true g estimated true x3 x3 x3 Figure 5: Estimated trasformatios for the first three variables. Wisorizatio plays a sigificat role for the power trasformatio due to its high skewess. 2309

16 LIU, LAFFERTY, AND WASSERMAN cdf power liear Oracle Score Oracle Score Oracle Score NoparaNormal Glasso NoparaNormal Glasso NoparaNormal Glasso Oracle Score Oracle Score Oracle Score NoparaNormal Glasso NoparaNormal Glasso NoparaNormal Glasso Oracle Score Oracle Score Oracle Score NoparaNormal Glasso NoparaNormal Glasso NoparaNormal Glasso Figure 6: Boxplots of the oracle scores for = 000,500,200 top, ceter, bottom QUANTITATIVE COMARISON To evaluate the performace for structure estimatio quatitatively, we use false positive ad false egative rates. Let G = V,E be a p-dimesioal graph which has at most p 2 edges i which there are E = r edges, ad let Ĝ λ = V,Ê λ be a estimated graph usig the regularizatio parameter λ. The umber of false positives at λ is Fλ umber of edges i Ê λ ot i E The umber of false egatives at λ is defied as The oracle regularizatio level λ is the FNλ umber of edges i E ot i Ê λ. λ = arg mi{fλ+fnλ}. λ Λ The oracle score is Fλ + FNλ. Figure 6 shows boxplots of the oracle scores for the two methods, calculated usig 00 simulatios. 230

17 THE NONARANORMAL To illustrate the overall performace of these two methods over the full paths, ROC curves are show i Figure 7, usig FNλ, Fλ r p. 2 r The curves clearly show how the performace of both methods improves with sample size, ad that the oparaormal is superior to the Gaussia model i most cases. cdf power liear CDF Trasform ower Trasform No Trasform F Noparaormal glasso F Noparaormal glasso F Noparaormal glasso FN CDF Trasform FN ower Trasform FN No Trasform F Noparaormal glasso F Noparaormal glasso F Noparaormal glasso FN CDF Trasform FN ower Trasform FN No Trasform F Noparaormal glasso F Noparaormal glasso F Noparaormal glasso FN FN FN Figure 7: ROC curves for sample sizes = 000,500,200 top, middle, bottom. Let FE Fλ ad FNE FNλ, Tables, 2, ad 3 provide umerical comparisos of both methods o data sets with differet trasformatios, where we repeat the experimets 00 times ad report the average FE ad FNE values with the correspodig stadard deviatios. It s clear from the tables that the oparaormal achieves sigificatly smaller errors tha the glasso if the true distributio of the data is ot multivariate Gaussia ad achieves performace comparable to the glasso whe the true distributio is exactly multivariate Gaussia. Figure 8 shows typical rus for the cdf ad power trasformatios. It s clear that whe the glasso estimates the graph icorrectly, the mistakes iclude both false positives ad egatives. 23

18 LIU, LAFFERTY, AND WASSERMAN Noparaormal glasso FE sdfe FNE sdfne FE sdfe FNE sdfne Table : Quatitative compariso o the data set usig the cdf trasformatio. For both FE ad FNE, the oparaormal performs much better i geeral. Noparaormal glasso FE sdfe FNE sdfne FE sdfe FNE sdfne Table 2: Quatitative compariso o the data set usig the power trasformatio. For both FE ad FNE, the oparaormal performs much better i geeral COMARISON IN THE GAUSSIAN CASE The previous experimets idicate that the oparaormal works almost as well as the glasso i the Gaussia case. This iitially appears surprisig, sice a parametric method is expected to be more efficiet tha a oparametric method if the parametric assumptio is correct. To maifest this efficiecy loss, we coducted some experimets with very small ad relatively large p. For multivariate Gaussia models, Figure 9 shows results with, p,s = 50,40,/8,50,00,/5 232

19 THE NONARANORMAL Noparaormal glasso FE sdfe FNE sdfne FE sdfe FNE sdfne Table 3: Quatitative compariso o the data set without ay trasformatio. The two methods behave similarly, the glasso is slightly better. ad 30, 00, /5. From the mea ROC curves, we see that oparaormal does ideed behave worse tha the glasso, suggestig some efficiecy loss. However, from the correspodig boxplots, the efficiecy reductio is relatively isigificat THE CASE WHEN p Figure 0 shows results from a simulatio of the oparaormal usig cdf trasformatios with = 200, p = 500 ad sparsity level s = /40. The boxplot shows that the oparaormal outperforms the glasso. A typical ru of the regularizatio paths cofirms this coclusio, showig that the oparaormal path separates the relevat ad irrelevat dimesios very well. I cotrast, with the glasso the relevat variables are buried amog the irrelevat variables. 6.2 Gee Microarray Data I this study, we cosider a data set based o Affymetrix GeeChip microarrays for the plat Arabidopsis thaliaa, Wille et al., The sample size is = 8. The expressio levels for each chip are pre-processed by log-trasformatio ad stadardizatio. A subset of 40 gees from the isopreoid pathway are chose, ad we study the associatios amog them usig both the paraormal ad oparaormal models. Eve though these data are geerally treated as multivariate Gaussia i the previous aalysis Wille et al., 2004, our study shows that the results of the oparaormal ad the glasso are very differet over a wide rage of regularizatio parameters. This suggests the oparaormal could support differet scietific coclusios COMARISON OF THE REGULARIZATION ATHS We first compare the regularizatio paths of the two methods, i Figure. To geerate the paths, we select 50 regularizatio parameters o a evely spaced grid i the iterval [0.6,.2]. Although 233

20 LIU, LAFFERTY, AND WASSERMAN cdf power true graph, p = 40 oparaormal, p = 40 true graph, p = 40 oparaormal, p = 40 z z z z graphical lasso, p = 40 symmetric differece, p = 40 z z z z z z graphical lasso, p = 40 symmetric differece, p = 40 z z z z z z true graph, p = 40 oparaormal, p = 40 true graph, p = 40 oparaormal, p = 40 z z z z graphical lasso, p = 40 symmetric differece, p = 40 z z z z z z graphical lasso, p = 40 symmetric differece, p = 40 z z z z z Figure 8: Typical rus for the two methods for = 000 usig the cdf ad power trasformatios. The dashed black lies i the symmetric differece plots idicate edges foud by the glasso but ot the oparaormal, ad vice-versa for the solid red lies. z the paths for the two methods look similar, there are some subtle differeces. I particular, variables become ozero i a differet order, especially whe the regularizatio parameter is i the rage λ [0.2, 0.3]. As show below, these subtle differeces i the paths lead to differet model selectio behaviors COMARISON OF THE ESTIMATED GRAHS Figure 2 compares the estimated graphs for the two methods at several values of the regularizatio parameter λ i the rage [0.6,0.37]. For each λ, we show the estimated graph from the oparaormal i the first colum. I the secod colum we show the graph obtaied by scaig the full 234

### Chapter 7 Methods of Finding Estimators

Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

### Properties of MLE: consistency, asymptotic normality. Fisher information.

Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

### 3. Covariance and Correlation

Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

### I. Chi-squared Distributions

1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

### Modified Line Search Method for Global Optimization

Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

### 7. Sample Covariance and Correlation

1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

### Confidence Intervals for One Mean with Tolerance Probability

Chapter 421 Cofidece Itervals for Oe Mea with Tolerace Probability Itroductio This procedure calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) with

### Department of Computer Science, University of Otago

Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

### Output Analysis (2, Chapters 10 &11 Law)

B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

### Methods of Evaluating Estimators

Math 541: Statistical Theory II Istructor: Sogfeg Zheg Methods of Evaluatig Estimators Let X 1, X 2,, X be i.i.d. radom variables, i.e., a radom sample from f(x θ), where θ is ukow. A estimator of θ is

### THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

### NPTEL STRUCTURAL RELIABILITY

NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics

### Maximum Likelihood Estimators.

Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

### Definition. Definition. 7-2 Estimating a Population Proportion. Definition. Definition

7- stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio

### AMS 2000 subject classification. Primary 62G08, 62G20; secondary 62G99

VARIABLE SELECTION IN NONPARAMETRIC ADDITIVE MODELS Jia Huag 1, Joel L. Horowitz 2 ad Fegrog Wei 3 1 Uiversity of Iowa, 2 Northwester Uiversity ad 3 Uiversity of West Georgia Abstract We cosider a oparametric

### Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

### Chapter Gaussian Elimination

Chapter 04.06 Gaussia Elimiatio After readig this chapter, you should be able to:. solve a set of simultaeous liear equatios usig Naïve Gauss elimiatio,. lear the pitfalls of the Naïve Gauss elimiatio

### Confidence Intervals for One Mean

Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

### ORDERS OF GROWTH KEITH CONRAD

ORDERS OF GROWTH KEITH CONRAD Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really wat to uderstad their behavior It also helps you better grasp topics i calculus

### Asymptotic Growth of Functions

CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

### In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

### Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

### A probabilistic proof of a binomial identity

A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

### 3. Continuous Random Variables

Statistics ad probability: 3-1 3. Cotiuous Radom Variables A cotiuous radom variable is a radom variable which ca take values measured o a cotiuous scale e.g. weights, stregths, times or legths. For ay

### CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 8

CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 8 GENE H GOLUB 1 Positive Defiite Matrices A matrix A is positive defiite if x Ax > 0 for all ozero x A positive defiite matrix has real ad positive

### Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

### ON SMALL SAMPLE PROPERTIES OF PERMUTATION TESTS: A SIGNIFICANCE TEST FOR REGRESSION MODELS*

Kobe Uiversity Ecoomic Review 52 (2006) 27 ON SMALL SAMPLE PROPERTIES OF PERMUTATION TESTS: A SIGNIFICANCE TEST FOR REGRESSION MODELS* By HISASHI TANIZAKI I this paper, we cosider a oparametric permutatio

### Incremental calculation of weighted mean and variance

Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

### 0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

### Hypothesis testing. Null and alternative hypotheses

Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

### Alternatives To Pearson s and Spearman s Correlation Coefficients

Alteratives To Pearso s ad Spearma s Correlatio Coefficiets Floreti Smaradache Chair of Math & Scieces Departmet Uiversity of New Mexico Gallup, NM 8730, USA Abstract. This article presets several alteratives

### Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

### Convexity, Inequalities, and Norms

Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

### 13 Fast Fourier Transform (FFT)

13 Fast Fourier Trasform FFT) The fast Fourier trasform FFT) is a algorithm for the efficiet implemetatio of the discrete Fourier trasform. We begi our discussio oce more with the cotiuous Fourier trasform.

### Stat 104 Lecture 16. Statistics 104 Lecture 16 (IPS 6.1) Confidence intervals - the general concept

Statistics 104 Lecture 16 (IPS 6.1) Outlie for today Cofidece itervals Cofidece itervals for a mea, µ (kow σ) Cofidece itervals for a proportio, p Margi of error ad sample size Review of mai topics for

### 4.1 Sigma Notation and Riemann Sums

0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

### Section 7-3 Estimating a Population. Requirements

Sectio 7-3 Estimatig a Populatio Mea: σ Kow Key Cocept This sectio presets methods for usig sample data to fid a poit estimate ad cofidece iterval estimate of a populatio mea. A key requiremet i this sectio

### Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

### BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

### High-dimensional support union recovery in multivariate regression

High-dimesioal support uio recovery i multivariate regressio Guillaume Oboziski Departmet of Statistics UC Berkeley gobo@stat.berkeley.edu Marti J. Waiwright Departmet of Statistics Dept. of Electrical

### Hypothesis Tests Applied to Means

The Samplig Distributio of the Mea Hypothesis Tests Applied to Meas Recall that the samplig distributio of the mea is the distributio of sample meas that would be obtaied from a particular populatio (with

### An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

### 1 Introduction to reducing variance in Monte Carlo simulations

Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by

### Gregory Carey, 1998 Linear Transformations & Composites - 1. Linear Transformations and Linear Composites

Gregory Carey, 1998 Liear Trasformatios & Composites - 1 Liear Trasformatios ad Liear Composites I Liear Trasformatios of Variables Meas ad Stadard Deviatios of Liear Trasformatios A liear trasformatio

### THE HEIGHT OF q-binary SEARCH TREES

THE HEIGHT OF q-binary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average

### LECTURE 13: Cross-validation

LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

### Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

### Overview of some probability distributions.

Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

### 1. C. The formula for the confidence interval for a population mean is: x t, which was

s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

### LIMIT DISTRIBUTION FOR THE WEIGHTED RANK CORRELATION COEFFICIENT, r W

REVSTAT Statistical Joural Volume 4, Number 3, November 2006, 189 200 LIMIT DISTRIBUTION FOR THE WEIGHTED RANK CORRELATION COEFFICIENT, r W Authors: Joaquim F. Pito da Costa Dep. de Matemática Aplicada,

### Lecture 4: Cheeger s Inequality

Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

### Sequences and Series

CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

### 5 Boolean Decision Trees (February 11)

5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

### Normal Distribution.

Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

### Notes on Hypothesis Testing

Probability & Statistics Grishpa Notes o Hypothesis Testig A radom sample X = X 1,..., X is observed, with joit pmf/pdf f θ x 1,..., x. The values x = x 1,..., x of X lie i some sample space X. The parameter

### Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

### Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown

Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about

### 1 Correlation and Regression Analysis

1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

### PSYCHOLOGICAL STATISTICS

UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION B Sc. Cousellig Psychology (0 Adm.) IV SEMESTER COMPLEMENTARY COURSE PSYCHOLOGICAL STATISTICS QUESTION BANK. Iferetial statistics is the brach of statistics

### Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

### Class Meeting # 16: The Fourier Transform on R n

MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

### Module 4: Mathematical Induction

Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate

### Plug-in martingales for testing exchangeability on-line

Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

### Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

### SUPPORT UNION RECOVERY IN HIGH-DIMENSIONAL MULTIVARIATE REGRESSION 1

The Aals of Statistics 2011, Vol. 39, No. 1, 1 47 DOI: 10.1214/09-AOS776 Istitute of Mathematical Statistics, 2011 SUPPORT UNION RECOVERY IN HIGH-DIMENSIONAL MULTIVARIATE REGRESSION 1 BY GUILLAUME OBOZINSKI,

### Data Analysis and Statistical Behaviors of Stock Market Fluctuations

44 JOURNAL OF COMPUTERS, VOL. 3, NO. 0, OCTOBER 2008 Data Aalysis ad Statistical Behaviors of Stock Market Fluctuatios Ju Wag Departmet of Mathematics, Beijig Jiaotog Uiversity, Beijig 00044, Chia Email:

### ARTICLE IN PRESS. Statistics & Probability Letters ( ) A Kolmogorov-type test for monotonicity of regression. Cecile Durot

STAPRO 66 pp: - col.fig.: il ED: MG PROD. TYPE: COM PAGN: Usha.N -- SCAN: il Statistics & Probability Letters 2 2 2 2 Abstract A Kolmogorov-type test for mootoicity of regressio Cecile Durot Laboratoire

### , a Wishart distribution with n -1 degrees of freedom and scale matrix.

UMEÅ UNIVERSITET Matematisk-statistiska istitutioe Multivariat dataaalys D MSTD79 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multivariat dataaalys D, 5 poäg.. Assume that

### Economics 140A Confidence Intervals and Hypothesis Testing

Ecoomics 140A Cofidece Itervals ad Hypothesis Testig Obtaiig a estimate of a parameter is ot the al purpose of statistical iferece because it is highly ulikely that the populatio value of a parameter is

### Chapter 14 Nonparametric Statistics

Chapter 14 Noparametric Statistics A.K.A. distributio-free statistics! Does ot deped o the populatio fittig ay particular type of distributio (e.g, ormal). Sice these methods make fewer assumptios, they

### University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

### HIGH-DIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH NONCONVEXITY

The Aals of Statistics 2012, Vol. 40, No. 3, 1637 1664 DOI: 10.1214/12-AOS1018 Istitute of Mathematical Statistics, 2012 HIGH-DIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH

### Soving Recurrence Relations

Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

### AQA STATISTICS 1 REVISION NOTES

AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if

### 5: Introduction to Estimation

5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

### Distributions of Order Statistics

Chapter 2 Distributios of Order Statistics We give some importat formulae for distributios of order statistics. For example, where F k: (x)=p{x k, x} = I F(x) (k, k + 1), I x (a,b)= 1 x t a 1 (1 t) b 1

### Section IV.5: Recurrence Relations from Algorithms

Sectio IV.5: Recurrece Relatios from Algorithms Give a recursive algorithm with iput size, we wish to fid a Θ (best big O) estimate for its ru time T() either by obtaiig a explicit formula for T() or by

### SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

### PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

### COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S 2 CONTROL CHART FOR THE CHANGES IN A PROCESS

COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S CONTROL CHART FOR THE CHANGES IN A PROCESS Supraee Lisawadi Departmet of Mathematics ad Statistics, Faculty of Sciece ad Techoology, Thammasat

### Fourier Series and the Wave Equation Part 2

Fourier Series ad the Wave Equatio Part There are two big ideas i our work this week. The first is the use of liearity to break complicated problems ito simple pieces. The secod is the use of the symmetries

### Irreducible polynomials with consecutive zero coefficients

Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem

### Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

### Research Method (I) --Knowledge on Sampling (Simple Random Sampling)

Research Method (I) --Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact

### BASIC STATISTICS. Discrete. Mass Probability Function: P(X=x i ) Only one finite set of values is considered {x 1, x 2,...} Prob. t = 1.

BASIC STATISTICS 1.) Basic Cocepts: Statistics: is a sciece that aalyzes iformatio variables (for istace, populatio age, height of a basketball team, the temperatures of summer moths, etc.) ad attempts

### B1. Fourier Analysis of Discrete Time Signals

B. Fourier Aalysis of Discrete Time Sigals Objectives Itroduce discrete time periodic sigals Defie the Discrete Fourier Series (DFS) expasio of periodic sigals Defie the Discrete Fourier Trasform (DFT)

### MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

### Entropy of bi-capacities

Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

### 1 Computing the Standard Deviation of Sample Means

Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

### Confidence Intervals and Sample Size

8/7/015 C H A P T E R S E V E N Cofidece Itervals ad Copyright 015 The McGraw-Hill Compaies, Ic. Permissio required for reproductio or display. 1 Cofidece Itervals ad Outlie 7-1 Cofidece Itervals for the

### Case Study. Contingency Tables. Graphing Tabled Counts. Stacked Bar Graph

Case Study Cotigecy Tables Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 4 6, 2011 Case Study Example 9.3 begiig o page 213 of the text describes a experimet i which

### A Gentle Introduction to Algorithms: Part II

A Getle Itroductio to Algorithms: Part II Cotets of Part I:. Merge: (to merge two sorted lists ito a sigle sorted list.) 2. Bubble Sort 3. Merge Sort: 4. The Big-O, Big-Θ, Big-Ω otatios: asymptotic bouds

### Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.

### Review for College Algebra Final Exam

Review for College Algebra Fial Exam (Please remember that half of the fial exam will cover chapters 1-4. This review sheet covers oly the ew material, from chapters 5 ad 7.) 5.1 Systems of equatios i

### The analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection

The aalysis of the Courot oligopoly model cosiderig the subjective motive i the strategy selectio Shigehito Furuyama Teruhisa Nakai Departmet of Systems Maagemet Egieerig Faculty of Egieerig Kasai Uiversity

### The second difference is the sequence of differences of the first difference sequence, 2

Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

### Analyzing Longitudinal Data from Complex Surveys Using SUDAAN

Aalyzig Logitudial Data from Complex Surveys Usig SUDAAN Darryl Creel Statistics ad Epidemiology, RTI Iteratioal, 312 Trotter Farm Drive, Rockville, MD, 20850 Abstract SUDAAN: Software for the Statistical

### Lecture 10: Hypothesis testing and confidence intervals

Eco 514: Probability ad Statistics Lecture 10: Hypothesis testig ad cofidece itervals Types of reasoig Deductive reasoig: Start with statemets that are assumed to be true ad use rules of logic to esure