1 Example 1: Axis-aligned rectangles

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "1 Example 1: Axis-aligned rectangles"

Transcription

1 COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton wth VC-dmenson, reduced the problem of fndng a good PAClearnng algorthm to the problem of computng the VC-dmenson of a gven hypothess space. Recall that VC-dmeson s defned usng the noton of a shattered set,.e. a subset S of the doman such that Π H (S 2 S. In ths lecture, we compute the VC-dmenson of several hypothess spaces by computng the maxmum sze of a shattered set. 1 Example 1: Axs-algned rectangles Not all sets of four ponts are shattered. For example the followng arrangement s mpossble: - Fgure 1: An mpossble assgnment of /- to the data, as all rectangles that contan the outer three ponts (marked must also contan the one pont. However, ths s not suffcent to conclude that the VC-dmenson s at most three. Note that the followng set does shatter: Fgure 2: A set of four ponts that shatters, as there s an axs-algned rectangle that contans any gven subset of the ponts but contans no others. Therefore, the VC-dmenson s at least four. In fact, t s exactly four. Consder any set of fve dstnct ponts {v 1, v 2, v 3, v 4, v 5 } R 2. Consder a rectangle that contans the ponts wth maxmum x-coordnate, mnmum x-coordnate, maxmum y-coordnate, and mnmum y-coordnate. These ponts may not be dstnct. However, there are at most four such ponts. Call ths set of ponts S {v 1, v 2, v 3, v 4, v 5 }. Any axs-algned rectangle that

2 contans S must also contan all of the ponts v 1, v 2, v 3, v 4, and v 5. There s at least one v that s not n S, but stll must be n the rectangle. Therefore, the labelng that labels all vertces n S wth and v wth cannot be consstent wth any axs-algned rectangle. Ths means that there s no shattered set of sze 5, snce all possble labelngs of a shattered set must be realzed by some concept. By a smlar argument, we can show that the VC-dmenson of axs-algned rectangles n R n s 2n. By generalzng the approach for provng that the VC-dmenson of the postve half nterval learnng problem s 1, one can show that the VC-dmenson of n 1 dmensonal hyperplanes n R n that pass through the orgn s n. Ths concepts are nequaltes of the form w x > 0 for any fxed w R n and varable x R n. In ths case, concepts label ponts wth f they are one sde of a hyperplane and otherwse. 2 Other remarks on VC-dmenson In the cases mentoned prevously, note that the VC-dmenson s smlar to the number of parameters needed to specfy any partcular concept. In the case of axs-algned rectangles, for example, they are equal snce rectangles requre a left boundary, a rght boundary, a top boundary, and a bottom boundary. Unfortunately, ths smlarty does not always hold, although t often does. There are some hypothess spaces wth nfnte VC-dmenson that can be specfed wth one parameter. Note that f H s fnte, the VC-dmenson s at most log 2 H, as at least 2 r dstnct hypotheses must exst to shatter a set of sze r. For a hypothess space wth nfnte VC-dmenson, there s a set of sze m that s shattered for any m > 0. Therefore, Π H (m 2 m, whch we mentoned last class as an ndcaton of a class that s hard to learn. In the next secton, we wll show that all classes wth bounded VC-dmenson d have Π H (m O(m d, completng the descrpton of PAC-learnablty by VC-dmenson. 3 Sauer s Lemma Recall that ( n k n! (n k!k! f 0 k n and ( n k 0 f k < 0 or k > n. k and n are ntegers and n s nonnegatve for our purposes. Note that ( n k O(n k when k s regarded as a postve constant. We wll show the followng lemma, whch mmedately mples the desred result: Lemma 3.1 (Sauer s Lemma. Let H be a hypothess wth fnte VC-dmenson d. Then, Π H (m d ( m : Φ d (m Proof. We wll prove ths by nducton on m d. There are two base cases: Case 1 (m 0. There s only one possble assgnment of and to the empty set,.e. Π H (m 1 here. Note that Φ d (0 ( 0 0 ( ( 0 d 1, as desred. 2

3 Case 2 (d 0. Not even a sngle pont can be shattered n ths stuaton. Therefore, on any gven pont, all hypotheses have the same value. Therefore, there s only one possble hypothess and Π H (m 1. Ths agrees wth Φ, as Φ 0 (m ( m 0 1. Now, we wll prove the nducton step. For ths, we wll need Pascal s Identty, whch states that ( ( ( n n n 1 k k 1 k 1 for all ntegers n and k wth n 0. Consder a hypothess space H wth VC-dmenson d and a set of m examples S : {x 1, x 2,..., x m }. Let T : {x 1, x 2,..., x m 1 }. Form two hypothess spaces H 1 and H 2 on T as follows (an example s n Fgure 3. Let H 1 be the set of restrctons of hypotheses from H to T. Let h T denote the restrcton of h to T for h H,.e. the functon h T : T {, } such that h T (x h(x for all x T. An element ρ on T s added to H 2 f and only f there are two dstnct hypotheses h 1, h 2 H such that h 1 T h 2 T ρ. Note that Π H (S Π H1 (T Π H2 (T. What are the VC-dmensons of H 1 and H 2? Frst, note that the VC-dmenson of H 1 s at most d, as any shatterng set of sze d 1 n T s also a subset of S that s shattered by the elements of H, contradctng the fact that the VC-dmenson of H s d. Suppose that there s a set of sze d n T that s shattered by H 2. Snce every hypothess n H 2 s the restrcton of two dfferent hypotheses n H, x m can be added to the shattered set of sze d n T to obtan a set shattered by H of sze d 1. Ths s a contradcton, so the VC-dmenson of H 2 s at most d 1. By the nductve hypothess, Π H1 (m 1 Φ d (m 1. Smlarly, Π H2 (m 1 Φ d 1 (m 1. Combnng these two nequaltes shows that Π H (m Φ d (m 1 Φ d 1 (m 1 ( d ( m 1 d 1 ( m 1 j j0 ( m 1 d 1 (( m 1 0 ( m d 1 ( m 0 1 Φ d (m completng the nductve step. ( m 1 1 Often, the polynomal Φ d (m s hard to work wth. Instead, we often use the followng result: Lemma 3.2. Φ d (m (em/d d when m d 1. Proof. m d 1 mples that d m 1. Therefore, snce d n the summand, 3

4 H x 1 x 2 x 3 x 4 x H 1 x 1 x 2 x 3 x H 2 x 1 x 2 x 3 x Fgure 3: The constructon of H 1 and H 2 ( d m d d ( m d ( d ( m m ( 1 d m m e d Multplyng on both sdes by (m/d d on both sdes gves the desred result. Pluggng ths result nto the examples bound proven last class shows that ( ( 1 err(h O d ln m m d ln 1 δ We can also wrte ths n terms of the number of examples requred to learn: ( 1 m O (ln 1/δ d ln 1/ɛ ɛ Note that the number of examples requred to learn scales lnearly wth the VC-dmenson. 4 Lower bounds on learnng The bound proven n the prevous secton shows that the VC-dmenson of a hypothess space yelds an upper bound on the number of examples needed to learn. Lower bounds on the requred number of examples also exst. If the VC-dmenson of a hypothess space s d, there s a shattered set of sze d. Intutvely, any hypothess learned from a subset of sze at most d 1 cannot predct the value of the last element wth probablty better than 1/2. Ths suggests that at least Ω(d examples are requred to learn. In future classes, we wll prove the followng Theorem 4.1. For all learnng algorthms A, there s a concept c C and a dstrbuton D such that f A s gven m d/2 examples labeled by c and dstrbuted accordng to D, then Pr[err(h A > 1/8] 1 8 4

5 One can try to prove ths as follows. Choose a unform dstrbuton D on examples {z 1,..., z d } and run A on m d/2 examples. Call ths set of examples S. Label the elements of S arbtrarly wth and -. Suppose that c C s selected to be consstent wth all of the labels on S and c(x h A (x for all x / S. err D (h A 1 2 snce c agrees wth h A on at most (d/2/2 1/2 of the probablty mass of the doman, whch means that there s no PAC-learnng algorthm on d/2 examples. Ths proof s flawed, as c needs to be chosen before the examples. We wll dscuss a correct proof n future classes. 5

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

We are now ready to answer the question: What are the possible cardinalities for finite fields?

We are now ready to answer the question: What are the possible cardinalities for finite fields? Chapter 3 Fnte felds We have seen, n the prevous chapters, some examples of fnte felds. For example, the resdue class rng Z/pZ (when p s a prme) forms a feld wth p elements whch may be dentfed wth the

More information

Graph Theory and Cayley s Formula

Graph Theory and Cayley s Formula Graph Theory and Cayley s Formula Chad Casarotto August 10, 2006 Contents 1 Introducton 1 2 Bascs and Defntons 1 Cayley s Formula 4 4 Prüfer Encodng A Forest of Trees 7 1 Introducton In ths paper, I wll

More information

greatest common divisor

greatest common divisor 4. GCD 1 The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

PLANAR GRAPHS. Plane graph (or embedded graph) A graph that is drawn on the plane without edge crossing, is called a Plane graph

PLANAR GRAPHS. Plane graph (or embedded graph) A graph that is drawn on the plane without edge crossing, is called a Plane graph PLANAR GRAPHS Basc defntons Isomorphc graphs Two graphs G(V,E) and G2(V2,E2) are somorphc f there s a one-to-one correspondence F of ther vertces such that the followng holds: - u,v V, uv E, => F(u)F(v)

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

Generalizing the degree sequence problem

Generalizing the degree sequence problem Mddlebury College March 2009 Arzona State Unversty Dscrete Mathematcs Semnar The degree sequence problem Problem: Gven an nteger sequence d = (d 1,...,d n ) determne f there exsts a graph G wth d as ts

More information

v a 1 b 1 i, a 2 b 2 i,..., a n b n i.

v a 1 b 1 i, a 2 b 2 i,..., a n b n i. SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 455 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces we have studed thus far n the text are real vector spaces snce the scalars are

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2)

n + d + q = 24 and.05n +.1d +.25q = 2 { n + d + q = 24 (3) n + 2d + 5q = 40 (2) MATH 16T Exam 1 : Part I (In-Class) Solutons 1. (0 pts) A pggy bank contans 4 cons, all of whch are nckels (5 ), dmes (10 ) or quarters (5 ). The pggy bank also contans a con of each denomnaton. The total

More information

2.4 Bivariate distributions

2.4 Bivariate distributions page 28 2.4 Bvarate dstrbutons 2.4.1 Defntons Let X and Y be dscrete r.v.s defned on the same probablty space (S, F, P). Instead of treatng them separately, t s often necessary to thnk of them actng together

More information

Lossless Data Compression

Lossless Data Compression Lossless Data Compresson Lecture : Unquely Decodable and Instantaneous Codes Sam Rowes September 5, 005 Let s focus on the lossless data compresson problem for now, and not worry about nosy channel codng

More information

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6 PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has

More information

Math 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011

Math 31 Lesson Plan. Day 27: Fundamental Theorem of Finite Abelian Groups. Elizabeth Gillaspy. November 11, 2011 Math 31 Lesson Plan Day 27: Fundamental Theorem of Fnte Abelan Groups Elzabeth Gllaspy November 11, 2011 Supples needed: Colored chal Quzzes Homewor 4 envelopes: evals, HW, presentaton rubrcs, * probs

More information

New bounds in Balog-Szemerédi-Gowers theorem

New bounds in Balog-Szemerédi-Gowers theorem New bounds n Balog-Szemeréd-Gowers theorem By Tomasz Schoen Abstract We prove, n partcular, that every fnte subset A of an abelan group wth the addtve energy κ A 3 contans a set A such that A κ A and A

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.

More information

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2

6. EIGENVALUES AND EIGENVECTORS 3 = 3 2 EIGENVALUES AND EIGENVECTORS The Characterstc Polynomal If A s a square matrx and v s a non-zero vector such that Av v we say that v s an egenvector of A and s the correspondng egenvalue Av v Example :

More information

A Note on the Decomposition of a Random Sample Size

A Note on the Decomposition of a Random Sample Size A Note on the Decomposton of a Random Sample Sze Klaus Th. Hess Insttut für Mathematsche Stochastk Technsche Unverstät Dresden Abstract Ths note addresses some results of Hess 2000) on the decomposton

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

Math 131: Homework 4 Solutions

Math 131: Homework 4 Solutions Math 3: Homework 4 Solutons Greg Parker, Wyatt Mackey, Chrstan Carrck October 6, 05 Problem (Munkres 3.) Let {A n } be a sequence of connected subspaces of X such that A n \ A n+ 6= ; for all n. Then S

More information

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier

Lesson 2 Chapter Two Three Phase Uncontrolled Rectifier Lesson 2 Chapter Two Three Phase Uncontrolled Rectfer. Operatng prncple of three phase half wave uncontrolled rectfer The half wave uncontrolled converter s the smplest of all three phase rectfer topologes.

More information

Non-degenerate Hilbert Cubes in Random Sets

Non-degenerate Hilbert Cubes in Random Sets Journal de Théore des Nombres de Bordeaux 00 (XXXX), 000 000 Non-degenerate Hlbert Cubes n Random Sets par Csaba Sándor Résumé. Une légère modfcaton de la démonstraton du lemme des cubes de Szemeréd donne

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

8 Algorithm for Binary Searching in Trees

8 Algorithm for Binary Searching in Trees 8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the

More information

Formula of Total Probability, Bayes Rule, and Applications

Formula of Total Probability, Bayes Rule, and Applications 1 Formula of Total Probablty, Bayes Rule, and Applcatons Recall that for any event A, the par of events A and A has an ntersecton that s empty, whereas the unon A A represents the total populaton of nterest.

More information

Extending Probabilistic Dynamic Epistemic Logic

Extending Probabilistic Dynamic Epistemic Logic Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

More information

A FASTER EXTERNAL SORTING ALGORITHM USING NO ADDITIONAL DISK SPACE

A FASTER EXTERNAL SORTING ALGORITHM USING NO ADDITIONAL DISK SPACE 47 A FASTER EXTERAL SORTIG ALGORITHM USIG O ADDITIOAL DISK SPACE Md. Rafqul Islam +, Mohd. oor Md. Sap ++, Md. Sumon Sarker +, Sk. Razbul Islam + + Computer Scence and Engneerng Dscplne, Khulna Unversty,

More information

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60

x f(x) 1 0.25 1 0.75 x 1 0 1 1 0.04 0.01 0.20 1 0.12 0.03 0.60 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

Multicomponent Distillation

Multicomponent Distillation Multcomponent Dstllaton need more than one dstllaton tower, for n components, n-1 fractonators are requred Specfcaton Lmtatons The followng are establshed at the begnnng 1. Temperature, pressure, composton,

More information

Properties of American Derivative Securities

Properties of American Derivative Securities Capter 6 Propertes of Amercan Dervatve Securtes 6.1 Te propertes Defnton 6.1 An Amercan dervatve securty s a sequence of non-negatve random varables fg k g n k= suc tat eac G k s F k -measurable. Te owner

More information

Ring structure of splines on triangulations

Ring structure of splines on triangulations www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

More information

INSTITUT FÜR INFORMATIK

INSTITUT FÜR INFORMATIK INSTITUT FÜR INFORMATIK An EPTAS for schedulng jobs on unform processors: usng an MILP relaxaton wth a constant number of ntegral varables Klaus Jansen Bercht Nr. 1002 February 2010 CHRISTIAN-ALBRECHTS-UNIVERSITÄT

More information

Embedding lattices in the Kleene degrees

Embedding lattices in the Kleene degrees F U N D A M E N T A MATHEMATICAE 62 (999) Embeddng lattces n the Kleene degrees by Hsato M u r a k (Nagoya) Abstract. Under ZFC+CH, we prove that some lattces whose cardnaltes do not exceed ℵ can be embedded

More information

General Auction Mechanism for Search Advertising

General Auction Mechanism for Search Advertising General Aucton Mechansm for Search Advertsng Gagan Aggarwal S. Muthukrshnan Dávd Pál Martn Pál Keywords game theory, onlne auctons, stable matchngs ABSTRACT Internet search advertsng s often sold by an

More information

Chapter 7. Random-Variate Generation 7.1. Prof. Dr. Mesut Güneş Ch. 7 Random-Variate Generation

Chapter 7. Random-Variate Generation 7.1. Prof. Dr. Mesut Güneş Ch. 7 Random-Variate Generation Chapter 7 Random-Varate Generaton 7. Contents Inverse-transform Technque Acceptance-Rejecton Technque Specal Propertes 7. Purpose & Overvew Develop understandng of generatng samples from a specfed dstrbuton

More information

Supplementary material: Assessing the relevance of node features for network structure

Supplementary material: Assessing the relevance of node features for network structure Supplementary materal: Assessng the relevance of node features for network structure Gnestra Bancon, 1 Paolo Pn,, 3 and Matteo Marsl 1 1 The Abdus Salam Internatonal Center for Theoretcal Physcs, Strada

More information

HIGH DEGREES OF RANDOM RECURSIVE TREES

HIGH DEGREES OF RANDOM RECURSIVE TREES HIGH DEGREES OF RANDOM RECURSIVE TREES LOUIGI ADDARIO-BERRY AND LAURA ESLAVA Abstract. For n, let T n be a random recursve tree (RRT on the vertex set [n] = {,..., n}. Let deg Tn (v be the degree of vertex

More information

5. Simultaneous eigenstates: Consider two operators that commute: Â η = a η (13.29)

5. Simultaneous eigenstates: Consider two operators that commute:  η = a η (13.29) 5. Smultaneous egenstates: Consder two operators that commute: [ Â, ˆB ] = 0 (13.28) Let  satsfy the followng egenvalue equaton: Multplyng both sdes by ˆB  η = a η (13.29) ˆB [  η ] = ˆB [a η ] = a

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

1. Measuring association using correlation and regression

1. Measuring association using correlation and regression How to measure assocaton I: Correlaton. 1. Measurng assocaton usng correlaton and regresson We often would lke to know how one varable, such as a mother's weght, s related to another varable, such as a

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

HYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION

HYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION HYPOTHESIS TESTING OF PARAMETERS FOR ORDINARY LINEAR CIRCULAR REGRESSION Abdul Ghapor Hussn Centre for Foundaton Studes n Scence Unversty of Malaya 563 KUALA LUMPUR E-mal: ghapor@umedumy Abstract Ths paper

More information

Implementation of Deutsch's Algorithm Using Mathcad

Implementation of Deutsch's Algorithm Using Mathcad Implementaton of Deutsch's Algorthm Usng Mathcad Frank Roux The followng s a Mathcad mplementaton of Davd Deutsch's quantum computer prototype as presented on pages - n "Machnes, Logc and Quantum Physcs"

More information

Solutions to the exam in SF2862, June 2009

Solutions to the exam in SF2862, June 2009 Solutons to the exam n SF86, June 009 Exercse 1. Ths s a determnstc perodc-revew nventory model. Let n = the number of consdered wees,.e. n = 4 n ths exercse, and r = the demand at wee,.e. r 1 = r = r

More information

IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1

IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS 1 Nov Sad J. Math. Vol. 36, No. 2, 2006, 0-09 IMPROVEMENT OF CONVERGENCE CONDITION OF THE SQUARE-ROOT INTERVAL METHOD FOR MULTIPLE ZEROS Modrag S. Petkovć 2, Dušan M. Mloševć 3 Abstract. A new theorem concerned

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

More information

Nasdaq Iceland Bond Indices 01 April 2015

Nasdaq Iceland Bond Indices 01 April 2015 Nasdaq Iceland Bond Indces 01 Aprl 2015 -Fxed duraton Indces Introducton Nasdaq Iceland (the Exchange) began calculatng ts current bond ndces n the begnnng of 2005. They were a response to recent changes

More information

Figure 1. Inventory Level vs. Time - EOQ Problem

Figure 1. Inventory Level vs. Time - EOQ Problem IEOR 54 Sprng, 009 rof Leahman otes on Eonom Lot Shedulng and Eonom Rotaton Cyles he Eonom Order Quantty (EOQ) Consder an nventory tem n solaton wth demand rate, holdng ost h per unt per unt tme, and replenshment

More information

On entropy for mixtures of discrete and continuous variables

On entropy for mixtures of discrete and continuous variables On entropy for mxtures of dscrete and contnuous varables Chandra Nar Balaj Prabhakar Devavrat Shah Abstract Let X be a dscrete random varable wth support S and f : S S be a bjecton. hen t s wellknown that

More information

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006

Aryabhata s Root Extraction Methods. Abhishek Parakh Louisiana State University Aug 31 st 2006 Aryabhata s Root Extracton Methods Abhshek Parakh Lousana State Unversty Aug 1 st 1 Introducton Ths artcle presents an analyss of the root extracton algorthms of Aryabhata gven n hs book Āryabhatīya [1,

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

REGULAR MULTILINEAR OPERATORS ON C(K) SPACES

REGULAR MULTILINEAR OPERATORS ON C(K) SPACES REGULAR MULTILINEAR OPERATORS ON C(K) SPACES FERNANDO BOMBAL AND IGNACIO VILLANUEVA Abstract. The purpose of ths paper s to characterze the class of regular contnuous multlnear operators on a product of

More information

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence 1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh

More information

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness

State function: eigenfunctions of hermitian operators-> normalization, orthogonality completeness Schroednger equaton Basc postulates of quantum mechancs. Operators: Hermtan operators, commutators State functon: egenfunctons of hermtan operators-> normalzaton, orthogonalty completeness egenvalues and

More information

Product-Form Stationary Distributions for Deficiency Zero Chemical Reaction Networks

Product-Form Stationary Distributions for Deficiency Zero Chemical Reaction Networks Bulletn of Mathematcal Bology (21 DOI 1.17/s11538-1-9517-4 ORIGINAL ARTICLE Product-Form Statonary Dstrbutons for Defcency Zero Chemcal Reacton Networks Davd F. Anderson, Gheorghe Cracun, Thomas G. Kurtz

More information

J. Parallel Distrib. Comput.

J. Parallel Distrib. Comput. J. Parallel Dstrb. Comput. 71 (2011) 62 76 Contents lsts avalable at ScenceDrect J. Parallel Dstrb. Comput. journal homepage: www.elsever.com/locate/jpdc Optmzng server placement n dstrbuted systems n

More information

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem.

Production. 2. Y is closed A set is closed if it contains its boundary. We need this for the solution existence in the profit maximization problem. Producer Theory Producton ASSUMPTION 2.1 Propertes of the Producton Set The producton set Y satsfes the followng propertes 1. Y s non-empty If Y s empty, we have nothng to talk about 2. Y s closed A set

More information

Quantization Effects in Digital Filters

Quantization Effects in Digital Filters Quantzaton Effects n Dgtal Flters Dstrbuton of Truncaton Errors In two's complement representaton an exact number would have nfntely many bts (n general). When we lmt the number of bts to some fnte value

More information

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits

Linear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.

More information

Online Learning from Experts: Minimax Regret

Online Learning from Experts: Minimax Regret E0 370 tatstcal Learnng Theory Lecture 2 Nov 24, 20) Onlne Learnng from Experts: Mn Regret Lecturer: hvan garwal crbe: Nkhl Vdhan Introducton In the last three lectures we have been dscussng the onlne

More information

Nonlinear data mapping by neural networks

Nonlinear data mapping by neural networks Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal

More information

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo

More information

Some geometric probability problems involving the Eulerian numbers

Some geometric probability problems involving the Eulerian numbers Some geometrc probablty problems nvolvng the Euleran numbers Frank Schmdt Rodca Smon Department of Mathematcs The George Washngton Unversty Washngton, DC 20052 smon@math.gwu.edu Dedcated to Herb Wlf on

More information

Scalar and Vector Quantization

Scalar and Vector Quantization Scalar and Vector Quantzaton Máro A. T. Fgueredo, Departamento de Engenhara Electrotécnca e de Computadores, Insttuto Superor Técnco, Lsboa, Portugal maro.fgueredo@st.utl.pt November 2008 Quantzaton s

More information

The University of Texas at Austin. Austin, Texas 78712. December 1987. Abstract. programs in which operations of dierent processes mayoverlap.

The University of Texas at Austin. Austin, Texas 78712. December 1987. Abstract. programs in which operations of dierent processes mayoverlap. Atomc Semantcs of Nonatomc Programs James H. Anderson Mohamed G. Gouda Department of Computer Scences The Unversty of Texas at Austn Austn, Texas 78712 December 1987 Abstract We argue that t s possble,

More information

A Probabilistic Theory of Coherence

A Probabilistic Theory of Coherence A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

More information

CHAPTER 14 MORE ABOUT REGRESSION

CHAPTER 14 MORE ABOUT REGRESSION CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp

More information

Two-phase algorithms for the parametric shortest path problem

Two-phase algorithms for the parametric shortest path problem Two-phase algorthms for the parametrc shortest path problem Sourav Chakraborty 1, Eldar Fscher 1, Oded Lachsh 2, and Raphael Yuster 3 1 Department of Computer Scence, Technon, Hafa 32000, Israel {eldar,sourav}@cs.technon.ac.l

More information

Fisher Markets and Convex Programs

Fisher Markets and Convex Programs Fsher Markets and Convex Programs Nkhl R. Devanur 1 Introducton Convex programmng dualty s usually stated n ts most general form, wth convex objectve functons and convex constrants. (The book by Boyd and

More information

z(t) = z 1 (t) + t(z 2 z 1 ) z(t) = 1 + i + t( 2 3i (1 + i)) z(t) = 1 + i + t( 3 4i); 0 t 1

z(t) = z 1 (t) + t(z 2 z 1 ) z(t) = 1 + i + t( 2 3i (1 + i)) z(t) = 1 + i + t( 3 4i); 0 t 1 (4.): ontours. Fnd an admssble parametrzaton. (a). the lne segment from z + to z 3. z(t) z (t) + t(z z ) z(t) + + t( 3 ( + )) z(t) + + t( 3 4); t (b). the crcle jz j 4 traversed once clockwse startng at

More information

9.1 The Cumulative Sum Control Chart

9.1 The Cumulative Sum Control Chart Learnng Objectves 9.1 The Cumulatve Sum Control Chart 9.1.1 Basc Prncples: Cusum Control Chart for Montorng the Process Mean If s the target for the process mean, then the cumulatve sum control chart s

More information

Multivariate EWMA Control Chart

Multivariate EWMA Control Chart Multvarate EWMA Control Chart Summary The Multvarate EWMA Control Chart procedure creates control charts for two or more numerc varables. Examnng the varables n a multvarate sense s extremely mportant

More information

Section B9: Zener Diodes

Section B9: Zener Diodes Secton B9: Zener Dodes When we frst talked about practcal dodes, t was mentoned that a parameter assocated wth the dode n the reverse bas regon was the breakdown voltage, BR, also known as the peak-nverse

More information

The Power of Slightly More than One Sample in Randomized Load Balancing

The Power of Slightly More than One Sample in Randomized Load Balancing The Power of Slghtly More than One Sample n Randomzed oad Balancng e Yng, R. Srkant and Xaohan Kang Abstract In many computng and networkng applcatons, arrvng tasks have to be routed to one of many servers,

More information

PERRON FROBENIUS THEOREM

PERRON FROBENIUS THEOREM PERRON FROBENIUS THEOREM R. CLARK ROBINSON Defnton. A n n matrx M wth real entres m, s called a stochastc matrx provded () all the entres m satsfy 0 m, () each of the columns sum to one, m = for all, ()

More information

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

More information

Efficient Striping Techniques for Variable Bit Rate Continuous Media File Servers æ

Efficient Striping Techniques for Variable Bit Rate Continuous Media File Servers æ Effcent Strpng Technques for Varable Bt Rate Contnuous Meda Fle Servers æ Prashant J. Shenoy Harrck M. Vn Department of Computer Scence, Department of Computer Scences, Unversty of Massachusetts at Amherst

More information

Communication Networks II Contents

Communication Networks II Contents 8 / 1 -- Communcaton Networs II (Görg) -- www.comnets.un-bremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP

More information

Introduction to Regression

Introduction to Regression Introducton to Regresson Regresson a means of predctng a dependent varable based one or more ndependent varables. -Ths s done by fttng a lne or surface to the data ponts that mnmzes the total error. -

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

Upper Bounds on the Cross-Sectional Volumes of Cubes and Other Problems

Upper Bounds on the Cross-Sectional Volumes of Cubes and Other Problems Upper Bounds on the Cross-Sectonal Volumes of Cubes and Other Problems Ben Pooley March 01 1 Contents 1 Prelmnares 1 11 Introducton 1 1 Basc Concepts and Notaton Cross-Sectonal Volumes of Cubes (Hyperplane

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

PROFIT RATIO AND MARKET STRUCTURE

PROFIT RATIO AND MARKET STRUCTURE POFIT ATIO AND MAKET STUCTUE By Yong Yun Introducton: Industral economsts followng from Mason and Ban have run nnumerable tests of the relaton between varous market structural varables and varous dmensons

More information

AN OPTIMAL ALGORITHM FOR CONFLICT-FREE COLORING FOR TREE OF RINGS

AN OPTIMAL ALGORITHM FOR CONFLICT-FREE COLORING FOR TREE OF RINGS AN OPTIMAL ALGORITHM FOR CONFLICT-FREE COLORING FOR TREE OF RINGS Enollah Pra The Busness Tranng Center of TabrzIran pra_ep2006@yahoo.com ABSTRACT An optmal algorthm s presented about Conflct-Free Colorng

More information

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6)

Passive Filters. References: Barbow (pp 265-275), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6) Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called

More information

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets

The Magnetic Field. Concepts and Principles. Moving Charges. Permanent Magnets . The Magnetc Feld Concepts and Prncples Movng Charges All charged partcles create electrc felds, and these felds can be detected by other charged partcles resultng n electrc force. However, a completely

More information

VoIP over Multiple IEEE 802.11 Wireless LANs

VoIP over Multiple IEEE 802.11 Wireless LANs SUBMITTED TO IEEE TRANSACTIONS ON MOBILE COMPUTING 1 VoIP over Multple IEEE 80.11 Wreless LANs An Chan, Graduate Student Member, IEEE, Soung Chang Lew, Senor Member, IEEE Abstract IEEE 80.11 WLAN has hgh

More information

Week 6 Market Failure due to Externalities

Week 6 Market Failure due to Externalities Week 6 Market Falure due to Externaltes 1. Externaltes n externalty exsts when the acton of one agent unavodably affects the welfare of another agent. The affected agent may be a consumer, gvng rse to

More information

Performance Analysis and Coding Strategy of ECOC SVMs

Performance Analysis and Coding Strategy of ECOC SVMs Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.67-76 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School

More information

An Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems

An Analysis of Central Processor Scheduling in Multiprogrammed Computer Systems STAN-CS-73-355 I SU-SE-73-013 An Analyss of Central Processor Schedulng n Multprogrammed Computer Systems (Dgest Edton) by Thomas G. Prce October 1972 Techncal Report No. 57 Reproducton n whole or n part

More information

LETTER IMAGE RECOGNITION

LETTER IMAGE RECOGNITION LETTER IMAGE RECOGNITION 1. Introducton. 1. Introducton. Objectve: desgn classfers for letter mage recognton. consder accuracy and tme n takng the decson. 20,000 samples: Startng set: mages based on 20

More information

How Much to Bet on Video Poker

How Much to Bet on Video Poker How Much to Bet on Vdeo Poker Trstan Barnett A queston that arses whenever a gae s favorable to the player s how uch to wager on each event? Whle conservatve play (or nu bet nzes large fluctuatons, t lacks

More information

Support vector domain description

Support vector domain description Pattern Recognton Letters 20 (1999) 1191±1199 www.elsever.nl/locate/patrec Support vector doman descrpton Davd M.J. Tax *,1, Robert P.W. Dun Pattern Recognton Group, Faculty of Appled Scence, Delft Unversty

More information