# Nimble Algorithms for Cloud Computing. Ravi Kannan, Santosh Vempala and David Woodruff

Save this PDF as:

Size: px
Start display at page:

Download "Nimble Algorithms for Cloud Computing. Ravi Kannan, Santosh Vempala and David Woodruff"

## Transcription

1 Nimble Algorithms for Cloud Computing Ravi Kannan, Santosh Vempala and David Woodruff

2 Cloud computing Data is distributed arbitrarily on many servers Parallel algorithms: time Streaming algorithms: sublinear space Cloud Complexity: time, space and communication [Cormode-Muthukrishnan-Ye 2008] Nimble algorithm: polynomial time/space (as usual) and sublinear (ideally polylog) communication between servers.

3 Cloud vs Streaming } Streaming algorithms make small sketches of data } Nimble algorithms must communicate small sketches } Are they equivalent? Simple observation: } Communication in cloud = O(memory in streaming) [Daume-Philips-Saha-Venkatasubramanian12] } Is cloud computing more powerful?

4 Basic Problems on large data sets } Frequency moments } Counting copies of subgraphs (homomorphisms) } Low-rank approximation } Clustering } } Matchings } Flows } Linear programs

5 Streaming Lower Bounds Frequency moments: Given a vector of frequencies f=( f 1, f 2,, f n ) presented as a set of increments, estimate f k = i f i k to relative error ε. [Alon-Matias-Szegedy99, Indyk-Woodruff05]: θ ( n 1 2/k ) space (k =1,2 by random projection) Counting homomorphisms: Estimate #triangles, # C 4, # K r,r in a large graph G. Ω ( n 2 ) space lower bounds in streaming.

6 Streaming Lower Bounds Low-rank approximation: Given n x d matrix A, find A of rank k s.t. A A F (1+ε) A A k F [Clarkson-Woodruff09] Any streaming algorithm needs Ω((n+d)k log nd) space.

7 Frequency moments in the cloud } Lower bound via multi-player set disjointness. } t players have sets S 1, S 2,, S t, subsets of [n] } Problem: determine if sets are disjoint or have one element in common. } Thm: Communication needed = Ω( n/t log t ) bits.

8 Frequency moments in the cloud Thm. Communication needed to determine set disjointness of t sets is Ω( n/t log t ) bits. Consider s sets being either (i) completely disjoint or (ii) with one common element (each set is on one server) Then k th frequency moment is either n or n 1+ s k Suppose we have a factor 2 approximation for the k th moment. With s k = n+1, then we can distinguish these cases. Therefore, communication needed is Ω( s k 1 ).

9 Frequency moments in the cloud Thm. [Kannan-V.-Woodruff13] Estimating k th frequency moment on s servers takes O( s k / ε 2 ) words of communication, with O(b+ log n ) bits per word. } Lower bound is s k 1 } Previous bound: s k 1 ( log n /ε ) O(k) Zhang12] } streaming space complexity is n 1 2/k [Woodruff- Main idea of algorithm: sample elements within a server according to higher moments.

10 Warm-up: 2 servers, third moment Goal: estimate i ( u i + v i ) 3 1. Estimate i u i 3 2. Sample j w.p. p j = u j 3 / i u i 3 ; announce 3. Second server computes X= u j 2 v j / p j 4. Average over many samples. E(X)= i u i 2 v i

11 Warm-up: 2 servers, third moment Goal: estimate i ( u i + v i ) 3 p j = u j 3 / i u i 3 X= u j 2 v j / p j E(X)= i u i 2 v i Var(X) i: v i >0 ( u i 2 v i ) 2 / p i i u i 3 i u i v i 2 ( i u i 3 + v i 3 ) 2 So, O( 1/ ε 2 ) samples suffice.

12 Many servers, k th moment }

13 Many servers, k th moment Each server j: } Sample i w. prob p i = f ij k / t f tj k k th moment. according to } Every j sends f i j if j < j and f i j < f ij or j >j and f i j f ij } Server j computes X i = j=1 s f i j r j / p i

14 Many servers, k th moment Each server j: } Sample i w. prob p i = f ij k / t f tj k k th moment. according to } Every j sends f i j if j < j and f i j < f ij or j >j and f i j f ij } Server j computes X i = j=1 s f i j r j / p i Lemma. E(X)= R j j f ij r j ( i f ij k ) 2 and Var(X) Theorem follows as there are < s k terms in total.

15 Counting homomorphisms } How many copies of graph H in large graph G? } E.g., H = triangle, 4-cycle, complete bipartite etc. } Linear lower bounds for counting 4-cycles, triangles. } We assume an (arbitrary) partition of the vertices among servers.

16 Counting homomorphisms } To count number of paths of length 2, in a graph with degrees d 1, d 2,, d n, we need: t( K 1,2,G)= i=1 n ( d i 2 ) This is a polynomial in frequency moments! } #stars is t( K 1,r,G)= i=1 n ( d i r ) } #C4 s: let d ij is the number of common neighbors of i and j. Then, t( C 4,G)= i=1 n ( d ij 2 ) } # K a,b : let d S be the number of common neighbors of a set of vertices S. Then, t( K a,b,g)= S V, S =a ( d S b )

17 Low-rank approximation Given n x d matrix A partitioned arbitrarily as A= A 1 + A A s among s servers, find A of rank k s.t. A A F (1+ε)OPT. To avoid linear communication, on each server t, we leave a matrix A t, s.t. A = A 1 + A A s and is of rank k. How to compute these matrices?

18 Low-rank approximation in the cloud Thm. [KVW13]. Low-rank approximation of n x d matrix A partitioned arbitrarily among s servers takes O (skd) communication.

19 Warm-up: row partition } Full matrix A is n x d with n >> d. } Each server j has a subset of rows A j } Computes A j T A j and sends to server 1. } Server 1 computes B= j=1 s A j T A j and announces V, the top k eigenvectors of B. } Now each server j can compute A j V V T. } Total communication = O(s d 2 ).

20 Low-rank approximation: arbitrary partition } To extend this to arbitrary partitions, we use limitedindependence random projection. } Subspace embedding: matrix P of size O( d/ ε 2 ) n s.t. for any x R d, PAx =(1±ε) Ax. } Agree on projection P via a random seed } Each server computes P A t, sends to server 1. } Server 1 computes PA= t P A t and its top k right singular vectors V. } Project rows of A to V. } Total communication = O( s d 2 / ε 2 ).

21 Low-rank approximation: arbitrary partition } Agree on projection P via a random seed } Each server computes P A t, sends to server 1. } Server 1 computes PA= t P A t and its top k right singular vectors V. } Project rows of A to V. Thm. A AV V T (1+O(ε))OPT. Pf. Extend V to a basis v 1, v 2,, v d. Then, A AV V T F 2 = i=k+1 d A v i 2 (1+ε) 2 i=k+1 d PA v i 2. And, with u 1, u 2,, u d singular vectors of A, i=k+1 d PA v i 2 i=k+1 d PA u i 2 (1+ε) 2 i=k+1 d A u i 2 =(1+O(ε))OP T 2.

22 Low-rank approximation in the cloud To improve to O(skd), we use a subspace embedding up front, and observe that O(k)-wise independence suffices for the random projection matrix. } Agree on O( k/ε ) n matrix S and O( k/ ε 2 ) n matrix P. } Each server computes S A t and sends to server 1. } S1 computes SA= t S A t and an orthonormal basis U T for its row space. } Apply previous algorithm to AU.

23 K-means clustering } Find a set of k centers c 1, c 2,, c k that minimize i S Min j=1 k A i c j 2 } A near-optimal (i.e. 1+ε) solution could be very different! } So, cannot project up front to reduce dimension and approximately preserve distances.

24 K-means clustering } Kannan-Kumar condition: } Every pair of cluster centers are f(k) standard deviations apart. } variance : maximum over 1-d projections, of the average squared distance of a point to its center. (e.g. for Gaussian mixtures, max directional variance) } Thm. [Kannan-Kumar10]. Under this condition, projection to the top k principal components followed by the k-means iteration starting at an approximately optimal set of centers finds a nearly correct clustering. } Finds centers close to the optimal ones, so that the induced clustering is same for most point.

25 K-means clustering in the cloud } Points (rows) are partitioned among servers } Low-rank approximation to project to SVD space. } How to find a good starting set of centers? } Need a constant-factor approximation. } Thm [Chen]. There exists a small subset ( core ) s.t. the k- means value of this set (weighted) is within a constant factor of the k-means value of the full set of points (for any set of centers!). } Chen s algorithm can be made nimble. Thm. K-means clustering in the cloud achieves the Kannan- Kumar guarantee with O( d 2 + k 4 ) communication on s = O(1) servers.

26 Cloud computing: What problems have nimble algorithms? } Approximate flow/matching? } Linear programs } Which graph properties/parameters can be checked/estimated in the cloud? (e.g., planarity? expansion? small diameter?) } Other Optimization/Clustering/ } Learning problems [Balcan-Blum-Fine-Mansour12, Daume-Philips-Saha-Venkatasubramaian12] } Random partition of data? } Connection to property testing?

### CIS 700: algorithms for Big Data

CIS 700: algorithms for Big Data Lecture 6: Graph Sketching Slides at http://grigory.us/big-data-class.html Grigory Yaroslavtsev http://grigory.us Sketching Graphs? We know how to sketch vectors: v Mv

### Sketching as a Tool for Numerical Linear Algebra

Sketching as a Tool for Numerical Linear Algebra (Graph Sparsification) David P. Woodruff presented by Sepehr Assadi o(n) Big Data Reading Group University of Pennsylvania April, 2015 Sepehr Assadi (Penn)

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### CSE 494 CSE/CBS 598 (Fall 2007): Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye

CSE 494 CSE/CBS 598 Fall 2007: Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye 1 Introduction One important method for data compression and classification is to organize

### APPM4720/5720: Fast algorithms for big data. Gunnar Martinsson The University of Colorado at Boulder

APPM4720/5720: Fast algorithms for big data Gunnar Martinsson The University of Colorado at Boulder Course objectives: The purpose of this course is to teach efficient algorithms for processing very large

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### Cycles and clique-minors in expanders

Cycles and clique-minors in expanders Benny Sudakov UCLA and Princeton University Expanders Definition: The vertex boundary of a subset X of a graph G: X = { all vertices in G\X with at least one neighbor

### Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Part 2: Community Detection

Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

### Big Data & Scripting Part II Streaming Algorithms

Big Data & Scripting Part II Streaming Algorithms 1, 2, a note on sampling and filtering sampling: (randomly) choose a representative subset filtering: given some criterion (e.g. membership in a set),

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Network Algorithms for Homeland Security

Network Algorithms for Homeland Security Mark Goldberg and Malik Magdon-Ismail Rensselaer Polytechnic Institute September 27, 2004. Collaborators J. Baumes, M. Krishmamoorthy, N. Preston, W. Wallace. Partially

### MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

### Week 5 Integral Polyhedra

Week 5 Integral Polyhedra We have seen some examples 1 of linear programming formulation that are integral, meaning that every basic feasible solution is an integral vector. This week we develop a theory

### 10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

### 1 Determinants and the Solvability of Linear Systems

1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### Lecture Topic: Low-Rank Approximations

Lecture Topic: Low-Rank Approximations Low-Rank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### On Clusterings: Good, Bad and Spectral

On Clusterings: Good, Bad and Spectral RAVI KANNAN Yale University, New Haven, Connecticut AND SANTOSH VEMPALA AND ADRIAN VETTA M.I.T., Cambridge, Massachusetts Abstract. We motivate and develop a natural

### B490 Mining the Big Data. 2 Clustering

B490 Mining the Big Data 2 Clustering Qin Zhang 1-1 Motivations Group together similar documents/webpages/images/people/proteins/products One of the most important problems in machine learning, pattern

### The Dirichlet Unit Theorem

Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if

### Lecture 4 Online and streaming algorithms for clustering

CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Social Media Mining. Graph Essentials

Graph Essentials Graph Basics Measures Graph and Essentials Metrics 2 2 Nodes and Edges A network is a graph nodes, actors, or vertices (plural of vertex) Connections, edges or ties Edge Node Measures

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### We shall turn our attention to solving linear systems of equations. Ax = b

59 Linear Algebra We shall turn our attention to solving linear systems of equations Ax = b where A R m n, x R n, and b R m. We already saw examples of methods that required the solution of a linear system

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Algorithmic Aspects of Big Data. Nikhil Bansal (TU Eindhoven)

Algorithmic Aspects of Big Data Nikhil Bansal (TU Eindhoven) Algorithm design Algorithm: Set of steps to solve a problem (by a computer) Studied since 1950 s. Given a problem: Find (i) best solution (ii)

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### (67902) Topics in Theory and Complexity Nov 2, 2006. Lecture 7

(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition

### Load balancing in a heterogeneous computer system by self-organizing Kohonen network

Bull. Nov. Comp. Center, Comp. Science, 25 (2006), 69 74 c 2006 NCC Publisher Load balancing in a heterogeneous computer system by self-organizing Kohonen network Mikhail S. Tarkov, Yakov S. Bezrukov Abstract.

Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-means

### 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)

Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible

### Chapter 19. General Matrices. An n m matrix is an array. a 11 a 12 a 1m a 21 a 22 a 2m A = a n1 a n2 a nm. The matrix A has n row vectors

Chapter 9. General Matrices An n m matrix is an array a a a m a a a m... = [a ij]. a n a n a nm The matrix A has n row vectors and m column vectors row i (A) = [a i, a i,..., a im ] R m a j a j a nj col

### The Open University s repository of research publications and other research outputs

Open Research Online The Open University s repository of research publications and other research outputs The degree-diameter problem for circulant graphs of degree 8 and 9 Journal Article How to cite:

### Network coding for security and robustness

Network coding for security and robustness 1 Outline Network coding for detecting attacks Network management requirements for robustness Centralized versus distributed network management 2 Byzantine security

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Graph theoretic techniques in the analysis of uniquely localizable sensor networks

Graph theoretic techniques in the analysis of uniquely localizable sensor networks Bill Jackson 1 and Tibor Jordán 2 ABSTRACT In the network localization problem the goal is to determine the location of

### USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS

USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu

### Improving Experiments by Optimal Blocking: Minimizing the Maximum Within-block Distance

Improving Experiments by Optimal Blocking: Minimizing the Maximum Within-block Distance Michael J. Higgins Jasjeet Sekhon April 12, 2014 EGAP XI A New Blocking Method A new blocking method with nice theoretical

### Markov random fields and Gibbs measures

Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### CS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #18: Dimensionality Reduc7on

CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #18: Dimensionality Reduc7on Dimensionality Reduc=on Assump=on: Data lies on or near a low d- dimensional subspace Axes of this subspace

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013

Lecture 11 Shuanglin Shao October 2nd and 7th, 2013 Matrix determinants: addition. Determinants: multiplication. Adjoint of a matrix. Cramer s rule to solve a linear system. Recall that from the previous

### Unsupervised Data Mining (Clustering)

Unsupervised Data Mining (Clustering) Javier Béjar KEMLG December 01 Javier Béjar (KEMLG) Unsupervised Data Mining (Clustering) December 01 1 / 51 Introduction Clustering in KDD One of the main tasks in

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Sublinear Algorithms for Big Data. Part 4: Random Topics

Sublinear Algorithms for Big Data Part 4: Random Topics Qin Zhang 1-1 2-1 Topic 1: Compressive sensing Compressive sensing The model (Candes-Romberg-Tao 04; Donoho 04) Applicaitons Medical imaging reconstruction

### Approximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs

Approximated Distributed Minimum Vertex Cover Algorithms for Bounded Degree Graphs Yong Zhang 1.2, Francis Y.L. Chin 2, and Hing-Fung Ting 2 1 College of Mathematics and Computer Science, Hebei University,

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Data Mining. Cluster Analysis: Advanced Concepts and Algorithms

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototype-based clustering Density-based clustering Graph-based

### Efficient Recovery of Secrets

Efficient Recovery of Secrets Marcel Fernandez Miguel Soriano, IEEE Senior Member Department of Telematics Engineering. Universitat Politècnica de Catalunya. C/ Jordi Girona 1 i 3. Campus Nord, Mod C3,

### 1 0 5 3 3 A = 0 0 0 1 3 0 0 0 0 0 0 0 0 0 0

Solutions: Assignment 4.. Find the redundant column vectors of the given matrix A by inspection. Then find a basis of the image of A and a basis of the kernel of A. 5 A The second and third columns are

### Permutation Betting Markets: Singleton Betting with Extra Information

Permutation Betting Markets: Singleton Betting with Extra Information Mohammad Ghodsi Sharif University of Technology ghodsi@sharif.edu Hamid Mahini Sharif University of Technology mahini@ce.sharif.edu

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM

International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 34-48 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT

### Well-Separated Pair Decomposition for the Unit-disk Graph Metric and its Applications

Well-Separated Pair Decomposition for the Unit-disk Graph Metric and its Applications Jie Gao Department of Computer Science Stanford University Joint work with Li Zhang Systems Research Center Hewlett-Packard

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### A Tutorial on Spectral Clustering

A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7276 Tübingen, Germany ulrike.luxburg@tuebingen.mpg.de This article appears in Statistics

### Lecture 6 Online and streaming algorithms for clustering

CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### A linear algebraic method for pricing temporary life annuities

A linear algebraic method for pricing temporary life annuities P. Date (joint work with R. Mamon, L. Jalen and I.C. Wang) Department of Mathematical Sciences, Brunel University, London Outline Introduction

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS

NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS DO Q LEE Abstract. Computing the solution to Least Squares Problems is of great importance in a wide range of fields ranging from numerical

### Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round \$200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

### Large induced subgraphs with all degrees odd

Large induced subgraphs with all degrees odd A.D. Scott Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, England Abstract: We prove that every connected graph of order

### Testing Hereditary Properties of Non-Expanding Bounded-Degree Graphs

Testing Hereditary Properties of Non-Expanding Bounded-Degree Graphs Artur Czumaj Asaf Shapira Christian Sohler Abstract We study graph properties which are testable for bounded degree graphs in time independent

### 3D Tranformations. CS 4620 Lecture 6. Cornell CS4620 Fall 2013 Lecture 6. 2013 Steve Marschner (with previous instructors James/Bala)

3D Tranformations CS 4620 Lecture 6 1 Translation 2 Translation 2 Translation 2 Translation 2 Scaling 3 Scaling 3 Scaling 3 Scaling 3 Rotation about z axis 4 Rotation about z axis 4 Rotation about x axis

### COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH. 1. Introduction

COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH ZACHARY ABEL 1. Introduction In this survey we discuss properties of the Higman-Sims graph, which has 100 vertices, 1100 edges, and is 22 regular. In fact

### Rank one SVD: un algorithm pour la visualisation d une matrice non négative

Rank one SVD: un algorithm pour la visualisation d une matrice non négative L. Labiod and M. Nadif LIPADE - Universite ParisDescartes, France ECAIS 2013 November 7, 2013 Outline Outline 1 Data visualization

### CITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION

No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August

### 17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function

17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function, : V V R, which is symmetric, that is u, v = v, u. bilinear, that is linear (in both factors):

### Topological Data Analysis Applications to Computer Vision

Topological Data Analysis Applications to Computer Vision Vitaliy Kurlin, http://kurlin.org Microsoft Research Cambridge and Durham University, UK Topological Data Analysis quantifies topological structures

### Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

### Random graphs with a given degree sequence

Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### Foundations of Data Science 1

Foundations of Data Science John Hopcroft Ravindran Kannan Version /4/204 These notes are a first draft of a book being written by Hopcroft and Kannan and in many places are incomplete. However, the notes

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### The Butterfly, Cube-Connected-Cycles and Benes Networks

The Butterfly, Cube-Connected-Cycles and Benes Networks Michael Lampis mlambis@softlab.ntua.gr NTUA The Butterfly, Cube-Connected-Cycles and Benes Networks p.1/16 Introduction Hypercubes are computationally

### Learning, Sparsity and Big Data

Learning, Sparsity and Big Data M. Magdon-Ismail (Joint Work) January 22, 2014. Out-of-Sample is What Counts NO YES A pattern exists We don t know it We have data to learn it Tested on new cases? Teaching

### Common Patterns and Pitfalls for Implementing Algorithms in Spark. Hossein Falaki @mhfalaki hossein@databricks.com

Common Patterns and Pitfalls for Implementing Algorithms in Spark Hossein Falaki @mhfalaki hossein@databricks.com Challenges of numerical computation over big data When applying any algorithm to big data

### APPLICATIONS OF MATRICES. Adj A is nothing but the transpose of the co-factor matrix [A ij ] of A.

APPLICATIONS OF MATRICES ADJOINT: Let A = [a ij ] be a square matrix of order n. Let Aij be the co-factor of a ij. Then the n th order matrix [A ij ] T is called the adjoint of A. It is denoted by adj

### 3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices.

Exercise 1 1. Let A be an n n orthogonal matrix. Then prove that (a) the rows of A form an orthonormal basis of R n. (b) the columns of A form an orthonormal basis of R n. (c) for any two vectors x,y R

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 3 Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### . P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2

4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### Going Big in Data Dimensionality:

LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DEPARTMENT INSTITUTE FOR INFORMATICS DATABASE Going Big in Data Dimensionality: Challenges and Solutions for Mining High Dimensional Data Peer Kröger Lehrstuhl für

### L15: statistical clustering

Similarity measures Criterion functions Cluster validity Flat clustering algorithms k-means ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern