The Intersection of Statistics and Topology:
|
|
- Cathleen Hancock
- 7 years ago
- Views:
Transcription
1 The Intersection of Statistics and Topology: Confidence Sets Brittany Terese Fasy joint work with S. Balakrishnan, F. Chazal, F. Lecci, A. Rinaldo, A. Singh, L. Wasserman 18 January 2014 B. Fasy (Tulane) Statistics and Topology 18 January / 31
2 Introduction How do we Interpret Data? Data can be a fininte subset of R D. What is the homology / the structure of the underlying space? B. Fasy (Tulane) Statistics and Topology 18 January / 31
3 Introduction How do we Interpret Data? Induced Topological Space B. Fasy (Tulane) Statistics and Topology 18 January / 31
4 Introduction How do we Interpret Data? Induced Topological Space B. Fasy (Tulane) Statistics and Topology 18 January / 31
5 Introduction How do we Interpret Data? Induced Topological Space B. Fasy (Tulane) Statistics and Topology 18 January / 31
6 Introduction How do we Interpret Data? Induced Topological Space B. Fasy (Tulane) Statistics and Topology 18 January / 31
7 Objective Introduction Let P be an unknown persistence diagram and P be an estimate of P. B. Fasy (Tulane) Statistics and Topology 18 January / 31
8 Objective Introduction Let P be an unknown persistence diagram and P be an estimate of P. Question How close is P to P? B. Fasy (Tulane) Statistics and Topology 18 January / 31
9 Introduction Objective Let P be an unknown persistence diagram and P be an estimate of P. Question How close is P to P? Answer with Statistics Given α (0, 1), we want δ α such that P(P {P : W (P, P) < δ α }) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
10 Introduction Objective Let P be an unknown persistence diagram and P be an estimate of P. Question How close is P to P? Answer with Statistics Given α (0, 1), we want δ α such that P(P {P : W (P, P) < δ α }) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
11 Statistical Model Background Statistics M is a manifold. P is a probability distribution supported on M. Observe data X 1, X 2,..., X n P. Compute ˆΘ n = Θ(X 1,..., X n ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
12 Statistical Model Background Statistics M is a manifold. P is a probability distribution supported on M. Observe data X 1, X 2,..., X n P. Compute ˆΘ n = Θ(X 1,..., X n ) Question How does ˆΘ n compare to E(Θ n ) = Θ n (M)? B. Fasy (Tulane) Statistics and Topology 18 January / 31
13 Background Statistics Statistical Model M is a manifold. P is a probability distribution supported on M. Observe data X 1, X 2,..., X n P. Compute ˆΘ n = Θ(X 1,..., X n ) Question How does ˆΘ n compare to E(Θ n ) = Θ n (M)? Answer Find C such that P(Θ n (M) C) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
14 Background Statistics Statistical Model M is a manifold. P is a probability distribution supported on M. Observe data X 1, X 2,..., X n P. Compute ˆΘ n = Θ(X 1,..., X n ) Question How does ˆΘ n compare to E(Θ n ) = Θ n (M)? Answer Find C such that P(Θ n (M) C) 1 α. How? B. Fasy (Tulane) Statistics and Topology 18 January / 31
15 Background Statistics Computing a Confidence Interval With Infinite Resources Repeatedly sample n data points, obtaining: Confidence Intervals P(Θ n (M) [0, q α ]) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
16 Background Statistics Computing a Confidence Interval With Infinite Resources Repeatedly sample n data points, obtaining: ˆΘ n,1,..., ˆΘ n,n Confidence Intervals P(Θ n (M) [0, q α ]) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
17 Background Statistics Computing a Confidence Interval With Infinite Resources Repeatedly sample n data points, obtaining: ˆΘ n,1,..., ˆΘ n,n via simulation. Confidence Intervals P(Θ n (M) [0, q α ]) 1 α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
18 Background Statistics Bootstrapping When We Can Only Take One Sample We have one sample: S n = {X 1,..., X n } B. Fasy (Tulane) Statistics and Topology 18 January / 31
19 Background Statistics Bootstrapping When We Can Only Take One Sample We have one sample: S n = {X 1,..., X n } Subsample (with replacement), obtaining: {X 1,..., X n } B. Fasy (Tulane) Statistics and Topology 18 January / 31
20 Background Statistics Bootstrapping When We Can Only Take One Sample We have one sample: S n = {X 1,..., X n } Subsample (with replacement), obtaining: {X 1,..., X n } Compute ˆΘ n = Θ(X 1,..., X n ). B. Fasy (Tulane) Statistics and Topology 18 January / 31
21 Background Statistics Bootstrapping When We Can Only Take One Sample We have one sample: S n = {X 1,..., X n } Subsample (with replacement), obtaining: {X 1,..., X n } Compute ˆΘ n = Θ(X 1,..., X n ). Repeat N times, obtaining: ˆΘ n,1,..., ˆΘ n,n. B. Fasy (Tulane) Statistics and Topology 18 January / 31
22 Background Statistics Bootstrapping When We Can Only Take One Sample We have one sample: S n = {X 1,..., X n } Subsample (with replacement), obtaining: {X 1,..., X n } Compute ˆΘ n = Θ(X 1,..., X n ). Repeat N times, obtaining: ˆΘ n,1,..., ˆΘ n,n. B. Fasy (Tulane) Statistics and Topology 18 January / 31
23 Bootstrapping Example Estimating Densities Background Statistics P has density p. Smoothed Density: p h = p K h KDE: ˆp h (x) = 1 n n 1 1 K h D ( x Xi h ). B. Fasy (Tulane) Statistics and Topology 18 January / 31
24 Bootstrapping Example Estimating Densities Background Statistics P has density p. Smoothed Density: p h = p K h KDE: ˆp h (x) = 1 n n 1 1 K h D ( x Xi h ). Θ n = ( nh D ˆp h p h ). Θ n = ( nh D ˆp h ˆp h ). B. Fasy (Tulane) Statistics and Topology 18 January / 31
25 Bootstrapping Example Estimating Densities Background Statistics P has density p. Smoothed Density: p h = p K h KDE: ˆp h (x) = 1 n n 1 1 K h D ( x Xi h ). Θ n = ( nh D ˆp h p h ). Θ n = ( nh D ˆp h ˆp h ). Bootstrap Theorem [FLRWBS] P( ( ) nh D ˆp h p h > q α X 1,..., X n ) = α + O 1/n B. Fasy (Tulane) Statistics and Topology 18 January / 31
26 Persistent Homology A Pairing of Critical Values. Background Topology P = Dgm + p (f ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
27 Persistent Homology A Pairing of Critical Values. Background Topology Tracking H ( f 1 ([t, )) ). P = Dgm + p (f ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
28 Persistent Homology A Pairing of Critical Values. Background Topology Tracking H ( f 1 ([t, )) ). P = Dgm + p (f ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
29 Persistent Homology A Pairing of Critical Values. Background Topology Tracking H ( f 1 ([t, )) ). P = Dgm + p (f ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
30 Bottleneck Distance Background Topology Given two persistence diagrams P and ˆP, find the best perfect matching between the point sets. Minimize Cost We wish to find W = min M { max (p,q) M p q }. B. Fasy (Tulane) Statistics and Topology 18 January / 31
31 Stability of Matchings Background Topology Bottleneck Stability Theorem [CDGO] p ˆp W (P, P) B. Fasy (Tulane) Statistics and Topology 18 January / 31
32 Putting It All Together Methods Bottleneck Stability Theorem p ˆp W (P, P) B. Fasy (Tulane) Statistics and Topology 18 January / 31
33 Putting It All Together Methods Bottleneck Stability Theorem p ˆp W (P, P) Bootstrap Theorem P( nh D ˆp h p h > q α ) = ( ) α + O 1/n B. Fasy (Tulane) Statistics and Topology 18 January / 31
34 Putting It All Together Methods Bottleneck Stability Theorem p ˆp W (P, P) Bootstrap Theorem P( nh D ˆp h p h > q α ) = ( ) α + O 1/n Confidence Sets for Persistence Diagrams P(W (P, P) qα ( ) ) 1 α O 1/n nh D B. Fasy (Tulane) Statistics and Topology 18 January / 31
35 Putting It All Together Methods Bottleneck Stability Theorem p ˆp W (P, P) Bootstrap Theorem P( nh D ˆp h p h > q α ) = ( ) α + O 1/n Confidence Sets for Persistence Diagrams P(W (P, P) qα ( ) ) 1 α O 1/n nh D Asymptotic Confidence Sets for Persistence Diagrams lim P(W (P, P) n qα nh D ) 1 α B. Fasy (Tulane) Statistics and Topology 18 January / 31
36 Methods Visualizing Confidence Intervals B. Fasy (Tulane) Statistics and Topology 18 January / 31
37 Methods Visualizing Confidence Intervals B. Fasy (Tulane) Statistics and Topology 18 January / 31
38 Methods Density Function Examples Uniform Distribution on Unit Circle Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
39 Methods Density Function Examples Uniform Distribution on Unit Circle Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
40 Methods Density Function Examples Uniform Distribution on Cassini Curve Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
41 Methods Density Function Examples Uniform Distribution on Cassini Curve Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
42 Methods Density Function Examples Cassini Curve with Outliers Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
43 Methods Density Function Examples Cassini Curve with Outliers Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
44 Methods Density Function Examples Normal Distribution on Unit Circle Results for Density Functions B. Fasy (Tulane) Statistics and Topology 18 January / 31
45 Methods Density Function Examples Normal Distribution on Unit Circle Results for Density Functions ] B. Fasy (Tulane) Statistics and Topology 18 January / 31
46 Distance to a Subset Methods Distance Function d M (a) = inf x M x a P 1 = Dgm p (d X ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
47 Distance to a Subset Methods Distance Function d M (a) = inf x M x a P 1 = Dgm p (d X ) P has continuous density p. support(p) = M. S n = {X 1,..., X n } P P 1 = Dgm p (d Sn ) B. Fasy (Tulane) Statistics and Topology 18 January / 31
48 Subsampling Methods Distance Function Confidence Interval from Subsampling [FLRWBS] Assume that p(x) is bounded away from zero. Then, almost surely, for all large n, P (W (P 1, P ) 1 ) > c n α + 2d n log n + O ( ) bn log n n B. Fasy (Tulane) Statistics and Topology 18 January / 31
49 Varying α Methods Distance Function B. Fasy (Tulane) Statistics and Topology 18 January / 31
50 Varying α Methods Distance Function α = 0.001, 0.05, 0.25 B. Fasy (Tulane) Statistics and Topology 18 January / 31
51 Two More Methods Methods Distance Function S n = S 1,n S2,n. Theorem (Concentration of Measure) There exists ˆt cm = ˆt cm (α, d, n, S 1,n ) such that ( P (W (P 1, P ) (log ) ) n 1/d+2 1 ) > ˆt cm α + O. n Theorem (Method of Shells) There exists ˆt s = ˆt s (α, d, n, K, S 1,n ) such that ( P (W (P 1, P ) (log ) ) n 1/d+2 1 ) > ˆt s α + O. n B. Fasy (Tulane) Statistics and Topology 18 January / 31
52 Methods Distance Function These Methods are Different Concentration of Measure ˆt cm is found by solving the following for t: 2 d+1 ( ) t d exp ntd ˆρ 1,n = α. ˆρ 1,n 2 Shells ˆt s is found by solving the following for t: 2 d+1 t d ˆρ n ( ) ĝ(v) v exp nvtd ˆρ 1,n dv = α. 2 B. Fasy (Tulane) Statistics and Topology 18 January / 31
53 Methods Distance Function Examples Uniform Distribution on Unit Circle Distance Function B. Fasy (Tulane) Statistics and Topology 18 January / 31
54 Methods Distance Function Examples Uniform Distribution on Cassini Curve Distance Function B. Fasy (Tulane) Statistics and Topology 18 January / 31
55 Methods Distance Function Examples Cassini Curve with Outliers Distance Function B. Fasy (Tulane) Statistics and Topology 18 January / 31
56 Methods Distance Function Examples Normal Distribution on Unit Circle Distance Function B. Fasy (Tulane) Statistics and Topology 18 January / 31
57 Conclusion Summary Recalling the Problem Sample from a distribution on a manifold. B. Fasy (Tulane) Statistics and Topology 18 January / 31
58 Conclusion Summary Recalling the Problem Sample from a distribution on a manifold. Create sample function (distance or density). B. Fasy (Tulane) Statistics and Topology 18 January / 31
59 Conclusion Summary Recalling the Problem Sample from a distribution on a manifold. Create sample function (distance or density). Now, we have (unknown) P and (known) P n. B. Fasy (Tulane) Statistics and Topology 18 January / 31
60 Summary Conclusion Recalling the Problem Sample from a distribution on a manifold. Create sample function (distance or density). Now, we have (unknown) P and (known) P n. Find c n such that P (W (P, P ) n ) > c n α. B. Fasy (Tulane) Statistics and Topology 18 January / 31
61 Summary Conclusion Recalling the Problem Sample from a distribution on a manifold. Create sample function (distance or density). Now, we have (unknown) P and (known) P n. Find c n such that P (W (P, P ) n ) > c n α. The pair P n and [0, c n ] define a confidence set for P. B. Fasy (Tulane) Statistics and Topology 18 January / 31
62 Ongoing Research Conclusion B. Fasy (Tulane) Statistics and Topology 18 January / 31
63 Conclusion Ongoing Research Functional Analysis Confidence Bands for Landscapes joint w/ F. Chazal, F. Lecci, A. Rinaldo, L. Wasserman B. Fasy (Tulane) Statistics and Topology 18 January / 31
64 Conclusion Ongoing Research Functional Analysis Confidence Bands for Landscapes joint w/ F. Chazal, F. Lecci, A. Rinaldo, L. Wasserman Really Great Upcoming Talk Carola Wenk Map Construction & Comparison 3:30 Here! B. Fasy (Tulane) Statistics and Topology 18 January / 31
65 Collaborator Collage Conclusion B. Fasy (Tulane) Statistics and Topology 18 January / 31
66 Thank you! Conclusion Brittany Terese Fasy B. Fasy (Tulane) Statistics and Topology 18 January / 31
67 References References [CDGO] The Structure and Stability of Persistence Modules. ArXiv [CFLRSW] On the Bootstrap for Persistence Diagrams and Landscapes. Modeling and Analysis of Information Systems, 20:6 (Dec. 2013), [FLRWBS] Statistical Inference for Persistent Homology: Confidence Sets for Persistence Diagrams. ArXiv Tentatively accepted, Annals of Statistics. B. Fasy (Tulane) Statistics and Topology 18 January / 31
Introduction to the R package TDA
Introduction to the R package TDA Brittany T. Fasy, Jisu Kim, Fabrizio Lecci, Clément Maria, and Vincent Rouvreau In collaboration with the CMU TopStat Group Abstract We present a short tutorial and introduction
More information15 Limit sets. Lyapunov functions
15 Limit sets. Lyapunov functions At this point, considering the solutions to ẋ = f(x), x U R 2, (1) we were most interested in the behavior of solutions when t (sometimes, this is called asymptotic behavior
More informationUn point de vue bayésien pour des algorithmes de bandit plus performants
Un point de vue bayésien pour des algorithmes de bandit plus performants Emilie Kaufmann, Telecom ParisTech Rencontre des Jeunes Statisticiens, Aussois, 28 août 2013 Emilie Kaufmann (Telecom ParisTech)
More informationFinite covers of a hyperbolic 3-manifold and virtual fibers.
Claire Renard Institut de Mathématiques de Toulouse November 2nd 2011 Some conjectures. Let M be a hyperbolic 3-manifold, connected, closed and oriented. Theorem (Kahn, Markovic) The fundamental group
More informationFast and Robust Normal Estimation for Point Clouds with Sharp Features
1/37 Fast and Robust Normal Estimation for Point Clouds with Sharp Features Alexandre Boulch & Renaud Marlet University Paris-Est, LIGM (UMR CNRS), Ecole des Ponts ParisTech Symposium on Geometry Processing
More informationTopological Data Analysis Applications to Computer Vision
Topological Data Analysis Applications to Computer Vision Vitaliy Kurlin, http://kurlin.org Microsoft Research Cambridge and Durham University, UK Topological Data Analysis quantifies topological structures
More informationMultivariate Normal Distribution
Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues
More informationThe sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].
Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationGeometry and Topology from Point Cloud Data
Geometry and Topology from Point Cloud Data Tamal K. Dey Department of Computer Science and Engineering The Ohio State University Dey (2011) Geometry and Topology from Point Cloud Data WALCOM 11 1 / 51
More informationRandom graphs with a given degree sequence
Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.
More informationQuantile Regression under misspecification, with an application to the U.S. wage structure
Quantile Regression under misspecification, with an application to the U.S. wage structure Angrist, Chernozhukov and Fernandez-Val Reading Group Econometrics November 2, 2010 Intro: initial problem The
More informationMATH 10550, EXAM 2 SOLUTIONS. x 2 + 2xy y 2 + x = 2
MATH 10550, EXAM SOLUTIONS (1) Find an equation for the tangent line to at the point (1, ). + y y + = Solution: The equation of a line requires a point and a slope. The problem gives us the point so we
More information8 Fractals: Cantor set, Sierpinski Triangle, Koch Snowflake, fractal dimension.
8 Fractals: Cantor set, Sierpinski Triangle, Koch Snowflake, fractal dimension. 8.1 Definitions Definition If every point in a set S has arbitrarily small neighborhoods whose boundaries do not intersect
More informationPrinciple of Data Reduction
Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then
More information1. Let A, B and C are three events such that P(A) = 0.45, P(B) = 0.30, P(C) = 0.35,
1. Let A, B and C are three events such that PA =.4, PB =.3, PC =.3, P A B =.6, P A C =.6, P B C =., P A B C =.7. a Compute P A B, P A C, P B C. b Compute P A B C. c Compute the probability that exactly
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationEstimating the Degree of Activity of jumps in High Frequency Financial Data. joint with Yacine Aït-Sahalia
Estimating the Degree of Activity of jumps in High Frequency Financial Data joint with Yacine Aït-Sahalia Aim and setting An underlying process X = (X t ) t 0, observed at equally spaced discrete times
More informationProbability and Statistics Vocabulary List (Definitions for Middle School Teachers)
Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationProduct Differentiation In homogeneous goods markets, price competition leads to perfectly competitive outcome, even with two firms Price competition
Product Differentiation In homogeneous goods markets, price competition leads to perfectly competitive outcome, even with two firms Price competition with differentiated products Models where differentiation
More informationTHE CENTRAL LIMIT THEOREM TORONTO
THE CENTRAL LIMIT THEOREM DANIEL RÜDT UNIVERSITY OF TORONTO MARCH, 2010 Contents 1 Introduction 1 2 Mathematical Background 3 3 The Central Limit Theorem 4 4 Examples 4 4.1 Roulette......................................
More informationSome stability results of parameter identification in a jump diffusion model
Some stability results of parameter identification in a jump diffusion model D. Düvelmeyer Technische Universität Chemnitz, Fakultät für Mathematik, 09107 Chemnitz, Germany Abstract In this paper we discuss
More informationMATH4427 Notebook 2 Spring 2016. 2 MATH4427 Notebook 2 3. 2.1 Definitions and Examples... 3. 2.2 Performance Measures for Estimators...
MATH4427 Notebook 2 Spring 2016 prepared by Professor Jenny Baglivo c Copyright 2009-2016 by Jenny A. Baglivo. All Rights Reserved. Contents 2 MATH4427 Notebook 2 3 2.1 Definitions and Examples...................................
More informationRECENT advances in digital technology have led to a
Robust, scalable and fast bootstrap method for analyzing large scale data Shahab Basiri, Esa Ollila, Member, IEEE, and Visa Koivunen, Fellow, IEEE arxiv:542382v2 [statme] 2 Apr 25 Abstract In this paper
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationErdős on polynomials
Erdős on polynomials Vilmos Totik University of Szeged and University of South Florida totik@mail.usf.edu Vilmos Totik (SZTE and USF) Polynomials 1 / * Erdős on polynomials Vilmos Totik (SZTE and USF)
More information14.11. Geodesic Lines, Local Gauss-Bonnet Theorem
14.11. Geodesic Lines, Local Gauss-Bonnet Theorem Geodesics play a very important role in surface theory and in dynamics. One of the main reasons why geodesics are so important is that they generalize
More informationA fast and robust algorithm to count topologically persistent holes in noisy clouds
A fast and robust algorithm to count topologically persistent holes in noisy clouds Vitaliy Kurlin Durham University Department of Mathematical Sciences, Durham, DH1 3LE, United Kingdom vitaliy.kurlin@gmail.com,
More informationHow To Find An Optimal Search Protocol For An Oblivious Cell
The Conference Call Search Problem in Wireless Networks Leah Epstein 1, and Asaf Levin 2 1 Department of Mathematics, University of Haifa, 31905 Haifa, Israel. lea@math.haifa.ac.il 2 Department of Statistics,
More information4. Expanding dynamical systems
4.1. Metric definition. 4. Expanding dynamical systems Definition 4.1. Let X be a compact metric space. A map f : X X is said to be expanding if there exist ɛ > 0 and L > 1 such that d(f(x), f(y)) Ld(x,
More informationA Numerical Study on the Wiretap Network with a Simple Network Topology
A Numerical Study on the Wiretap Network with a Simple Network Topology Fan Cheng and Vincent Tan Department of Electrical and Computer Engineering National University of Singapore Mathematical Tools of
More informationMA4001 Engineering Mathematics 1 Lecture 10 Limits and Continuity
MA4001 Engineering Mathematics 1 Lecture 10 Limits and Dr. Sarah Mitchell Autumn 2014 Infinite limits If f(x) grows arbitrarily large as x a we say that f(x) has an infinite limit. Example: f(x) = 1 x
More informationSemi-Supervised Support Vector Machines and Application to Spam Filtering
Semi-Supervised Support Vector Machines and Application to Spam Filtering Alexander Zien Empirical Inference Department, Bernhard Schölkopf Max Planck Institute for Biological Cybernetics ECML 2006 Discovery
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationLesson 2: Circles, Chords, Diameters, and Their Relationships
Circles, Chords, Diameters, and Their Relationships Student Outcomes Identify the relationships between the diameters of a circle and other chords of the circle. Lesson Notes Students are asked to construct
More informationSOLUTIONS. f x = 6x 2 6xy 24x, f y = 3x 2 6y. To find the critical points, we solve
SOLUTIONS Problem. Find the critical points of the function f(x, y = 2x 3 3x 2 y 2x 2 3y 2 and determine their type i.e. local min/local max/saddle point. Are there any global min/max? Partial derivatives
More informationAnomaly detection for Big Data, networks and cyber-security
Anomaly detection for Big Data, networks and cyber-security Patrick Rubin-Delanchy University of Bristol & Heilbronn Institute for Mathematical Research Joint work with Nick Heard (Imperial College London),
More informationDifferentiating under an integral sign
CALIFORNIA INSTITUTE OF TECHNOLOGY Ma 2b KC Border Introduction to Probability and Statistics February 213 Differentiating under an integral sign In the derivation of Maximum Likelihood Estimators, or
More informationMathematics Review for MS Finance Students
Mathematics Review for MS Finance Students Anthony M. Marino Department of Finance and Business Economics Marshall School of Business Lecture 1: Introductory Material Sets The Real Number System Functions,
More informationA linear algebraic method for pricing temporary life annuities
A linear algebraic method for pricing temporary life annuities P. Date (joint work with R. Mamon, L. Jalen and I.C. Wang) Department of Mathematical Sciences, Brunel University, London Outline Introduction
More informationMTH 140 Statistics Videos
MTH 140 Statistics Videos Chapter 1 Picturing Distributions with Graphs Individuals and Variables Categorical Variables: Pie Charts and Bar Graphs Categorical Variables: Pie Charts and Bar Graphs Quantitative
More informationIntroduction to Detection Theory
Introduction to Detection Theory Reading: Ch. 3 in Kay-II. Notes by Prof. Don Johnson on detection theory, see http://www.ece.rice.edu/~dhj/courses/elec531/notes5.pdf. Ch. 10 in Wasserman. EE 527, Detection
More information5.1 Derivatives and Graphs
5.1 Derivatives and Graphs What does f say about f? If f (x) > 0 on an interval, then f is INCREASING on that interval. If f (x) < 0 on an interval, then f is DECREASING on that interval. A function has
More informationBig Data: The Computation/Statistics Interface
Big Data: The Computation/Statistics Interface Michael I. Jordan University of California, Berkeley September 2, 2013 What Is the Big Data Phenomenon? Big Science is generating massive datasets to be used
More informationQuantile Inference and Statistical Analysis of Cancer
July 2003 QUANTILE PROBABILITY AND STATISTICAL DATA MODELING by Emanuel Parzen Texas A&M University ABSTRACT Quantile and conditional quantile statistical thinking, as I have innovated it in my research
More informationINFERENCE FOR THE IDENTIFIED SET IN PARTIALLY IDENTIFIED ECONOMETRIC MODELS. JOSEPH P. ROMANO Stanford University, Stanford, CA 94305-4065, U.S.A.
http://www.econometricsociety.org/ Econometrica, Vol. 78, No. 1 (January, 2010), 169 211 INFERENCE FOR THE IDENTIFIED SET IN PARTIALLY IDENTIFIED ECONOMETRIC MODELS JOSEPH P. ROMANO Stanford University,
More informationGRADES 7, 8, AND 9 BIG IDEAS
Table 1: Strand A: BIG IDEAS: MATH: NUMBER Introduce perfect squares, square roots, and all applications Introduce rational numbers (positive and negative) Introduce the meaning of negative exponents for
More informationUnderstanding Basic Calculus
Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other
More informationTESTING RESTRICTIONS IN NONPARAMETRIC EFFICIENCY MODELS
COMMUN. STATIST. SIMULA., 30(1), 159 184 (2001) TESTING RESTRICTIONS IN NONPARAMETRIC EFFICIENCY MODELS Léopold Simar 1, and Paul W. Wilson 2, 1 Institut de Statistique, Université Catholique de Louvain,
More informationChristfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic
More informationCoding Theorems for Turbo-Like Codes Abstract. 1. Introduction.
Coding Theorems for Turbo-Like Codes Dariush Divsalar, Hui Jin, and Robert J. McEliece Jet Propulsion Laboratory and California Institute of Technology Pasadena, California USA E-mail: dariush@shannon.jpl.nasa.gov,
More informationMigration with Local Public Goods and the Gains from Changing Places
Migration with Local Public Goods and the Gains from Changing Places Peter J. Hammond Department of Economics, University of Warwick, Coventry CV4 7AL, U.K. p.j.hammond@warwick.ac.edu Jaume Sempere C.E.E.,
More informationDongfeng Li. Autumn 2010
Autumn 2010 Chapter Contents Some statistics background; ; Comparing means and proportions; variance. Students should master the basic concepts, descriptive statistics measures and graphs, basic hypothesis
More informationInstitute of Actuaries of India Subject CT3 Probability and Mathematical Statistics
Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in
More informationLecture 4: BK inequality 27th August and 6th September, 2007
CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound
More informationGENERIC COMPUTABILITY, TURING DEGREES, AND ASYMPTOTIC DENSITY
GENERIC COMPUTABILITY, TURING DEGREES, AND ASYMPTOTIC DENSITY CARL G. JOCKUSCH, JR. AND PAUL E. SCHUPP Abstract. Generic decidability has been extensively studied in group theory, and we now study it in
More informationINDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS
INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem
More informationPhysics 210 Q1 2012 ( PHYSICS210BRIDGE ) My Courses Course Settings
1 of 11 9/7/2012 1:06 PM Logged in as Julie Alexander, Instructor Help Log Out Physics 210 Q1 2012 ( PHYSICS210BRIDGE ) My Courses Course Settings Course Home Assignments Roster Gradebook Item Library
More informationConsistent Binary Classification with Generalized Performance Metrics
Consistent Binary Classification with Generalized Performance Metrics Nagarajan Natarajan Joint work with Oluwasanmi Koyejo, Pradeep Ravikumar and Inderjit Dhillon UT Austin Nov 4, 2014 Problem and Motivation
More informationGeography 4203 / 5203. GIS Modeling. Class (Block) 9: Variogram & Kriging
Geography 4203 / 5203 GIS Modeling Class (Block) 9: Variogram & Kriging Some Updates Today class + one proposal presentation Feb 22 Proposal Presentations Feb 25 Readings discussion (Interpolation) Last
More informationCommon Core Unit Summary Grades 6 to 8
Common Core Unit Summary Grades 6 to 8 Grade 8: Unit 1: Congruence and Similarity- 8G1-8G5 rotations reflections and translations,( RRT=congruence) understand congruence of 2 d figures after RRT Dilations
More informationVisualization and descriptive statistics. D.A. Forsyth
Visualization and descriptive statistics D.A. Forsyth What s going on here? Most important, most creative scientific question Getting answers Make helpful pictures and look at them Compute numbers in support
More informationValuation of Lease Contracts In Continuous Time With Stochastic Asset Values
Valuation of Lease Contracts In Continuous Time With Stochastic Asset Values Riaz Hussain *+ * The University of Scranton Abstract A lease is a derivative security the value of which depends upon the value
More information6. Define log(z) so that π < I log(z) π. Discuss the identities e log(z) = z and log(e w ) = w.
hapter omplex integration. omplex number quiz. Simplify 3+4i. 2. Simplify 3+4i. 3. Find the cube roots of. 4. Here are some identities for complex conjugate. Which ones need correction? z + w = z + w,
More informationThe Gaussian Surface Area and Noise Sensitivity of Degree-d Polynomial Threshold Functions. Daniel M. Kane
The Gaussian Surface Area and Noise Sensitivity of Degree-d Polynomial Threshold Functions Daniel M. Kane Outline Definitions and Motivation Previous Work Statements of Theorems Proof of the Noise Sensitivity
More informationChapter 4 - Lecture 1 Probability Density Functions and Cumul. Distribution Functions
Chapter 4 - Lecture 1 Probability Density Functions and Cumulative Distribution Functions October 21st, 2009 Review Probability distribution function Useful results Relationship between the pdf and the
More informationHydrodynamic Limits of Randomized Load Balancing Networks
Hydrodynamic Limits of Randomized Load Balancing Networks Kavita Ramanan and Mohammadreza Aghajani Brown University Stochastic Networks and Stochastic Geometry a conference in honour of François Baccelli
More informationz-scores AND THE NORMAL CURVE MODEL
z-scores AND THE NORMAL CURVE MODEL 1 Understanding z-scores 2 z-scores A z-score is a location on the distribution. A z- score also automatically communicates the raw score s distance from the mean A
More informationLecture 7: Continuous Random Variables
Lecture 7: Continuous Random Variables 21 September 2005 1 Our First Continuous Random Variable The back of the lecture hall is roughly 10 meters across. Suppose it were exactly 10 meters, and consider
More informationMessage-passing sequential detection of multiple change points in networks
Message-passing sequential detection of multiple change points in networks Long Nguyen, Arash Amini Ram Rajagopal University of Michigan Stanford University ISIT, Boston, July 2012 Nguyen/Amini/Rajagopal
More informationSignal detection and goodness-of-fit: the Berk-Jones statistics revisited
Signal detection and goodness-of-fit: the Berk-Jones statistics revisited Jon A. Wellner (Seattle) INET Big Data Conference INET Big Data Conference, Cambridge September 29-30, 2015 Based on joint work
More informationNon Parametric Inference
Maura Department of Economics and Finance Università Tor Vergata Outline 1 2 3 Inverse distribution function Theorem: Let U be a uniform random variable on (0, 1). Let X be a continuous random variable
More informationEvaluation & Validation: Credibility: Evaluating what has been learned
Evaluation & Validation: Credibility: Evaluating what has been learned How predictive is a learned model? How can we evaluate a model Test the model Statistical tests Considerations in evaluating a Model
More informationConvergence rates for persistence diagram estimation in Topological Data Analysis
Convergence rates for persistence diagram estimation in Topological Data Analysis Frédéric Chazal INRIA Saclay Île-de-France, Palaiseau, France Marc Glisse INRIA Saclay Île-de-France, Palaiseau, France
More informationMATH 10: Elementary Statistics and Probability Chapter 5: Continuous Random Variables
MATH 10: Elementary Statistics and Probability Chapter 5: Continuous Random Variables Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of slides,
More informationSummary. Load and Open GaussView to start example
Summary This document describes in great detail how to navigate the Linux Red Hat Terminal to bring up GaussView, use GaussView to create a simple atomic or molecular simulation input file, and then use
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
More informationPublic Switched Telephone System
Public Switched Telephone System Structure of the Telephone System The Local Loop: Modems, ADSL Structure of the Telephone System (a) Fully-interconnected network. (b) Centralized switch. (c) Two-level
More informationProblem 1 (25 points)
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Physics 8.02 Spring 2012 Exam Three Solutions Problem 1 (25 points) Question 1 (5 points) Consider two circular rings of radius R, each perpendicular
More informationSections 2.11 and 5.8
Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and
More informationOn The Existence Of Flips
On The Existence Of Flips Hacon and McKernan s paper, arxiv alg-geom/0507597 Brian Lehmann, February 2007 1 Introduction References: Hacon and McKernan s paper, as above. Kollár and Mori, Birational Geometry
More informationA Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails
12th International Congress on Insurance: Mathematics and Economics July 16-18, 2008 A Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails XUEMIAO HAO (Based on a joint
More informationFixed Point Theorems
Fixed Point Theorems Definition: Let X be a set and let T : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation
More informationData Exploration Data Visualization
Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select
More informationSOLUTIONS TO ASSIGNMENT 1 MATH 576
SOLUTIONS TO ASSIGNMENT 1 MATH 576 SOLUTIONS BY OLIVIER MARTIN 13 #5. Let T be the topology generated by A on X. We want to show T = J B J where B is the set of all topologies J on X with A J. This amounts
More informationJUST THE MATHS UNIT NUMBER 8.5. VECTORS 5 (Vector equations of straight lines) A.J.Hobson
JUST THE MATHS UNIT NUMBER 8.5 VECTORS 5 (Vector equations of straight lines) by A.J.Hobson 8.5.1 Introduction 8.5. The straight line passing through a given point and parallel to a given vector 8.5.3
More informationStatistics 104: Section 6!
Page 1 Statistics 104: Section 6! TF: Deirdre (say: Dear-dra) Bloome Email: dbloome@fas.harvard.edu Section Times Thursday 2pm-3pm in SC 109, Thursday 5pm-6pm in SC 705 Office Hours: Thursday 6pm-7pm SC
More informationPRACTICE FINAL. Problem 1. Find the dimensions of the isosceles triangle with largest area that can be inscribed in a circle of radius 10cm.
PRACTICE FINAL Problem 1. Find the dimensions of the isosceles triangle with largest area that can be inscribed in a circle of radius 1cm. Solution. Let x be the distance between the center of the circle
More informationSchatten van de extreme value index voor afgeronde data (Engelse titel: Estimation of the extreme value index for imprecise data)
Technische Universiteit Delft Faculteit Elektrotechniek, Wiskunde en Informatica Delft Institute of Applied Mathematics Schatten van de extreme value index voor afgeronde data (Engelse titel: Estimation
More informationFuzzy Differential Systems and the New Concept of Stability
Nonlinear Dynamics and Systems Theory, 1(2) (2001) 111 119 Fuzzy Differential Systems and the New Concept of Stability V. Lakshmikantham 1 and S. Leela 2 1 Department of Mathematical Sciences, Florida
More informationDuality of linear conic problems
Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least
More informationSENSITIVITY ANALYSIS AND INFERENCE. Lecture 12
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationWhy do mathematicians make things so complicated?
Why do mathematicians make things so complicated? Zhiqin Lu, The Math Department March 9, 2010 Introduction What is Mathematics? Introduction What is Mathematics? 24,100,000 answers from Google. Introduction
More informationMA 323 Geometric Modelling Course Notes: Day 02 Model Construction Problem
MA 323 Geometric Modelling Course Notes: Day 02 Model Construction Problem David L. Finn November 30th, 2004 In the next few days, we will introduce some of the basic problems in geometric modelling, and
More informationR 2 -type Curves for Dynamic Predictions from Joint Longitudinal-Survival Models
Faculty of Health Sciences R 2 -type Curves for Dynamic Predictions from Joint Longitudinal-Survival Models Inference & application to prediction of kidney graft failure Paul Blanche joint work with M-C.
More informationON THE EMBEDDING OF BIRKHOFF-WITT RINGS IN QUOTIENT FIELDS
ON THE EMBEDDING OF BIRKHOFF-WITT RINGS IN QUOTIENT FIELDS DOV TAMARI 1. Introduction and general idea. In this note a natural problem arising in connection with so-called Birkhoff-Witt algebras of Lie
More informationFalse-Alarm and Non-Detection Probabilities for On-line Quality Control via HMM
Int. Journal of Math. Analysis, Vol. 6, 2012, no. 24, 1153-1162 False-Alarm and Non-Detection Probabilities for On-line Quality Control via HMM C.C.Y. Dorea a,1, C.R. Gonçalves a, P.G. Medeiros b and W.B.
More information