Tensor Methods for Machine Learning, Computer Vision, and Computer Graphics


 Mark Hart
 3 years ago
 Views:
Transcription
1 Tensor Methods for Machine Learning, Computer Vision, and Computer Graphics Part I: Factorizations and Statistical Modeling/Inference Amnon Shashua School of Computer Science & Eng. The Hebrew University ICML07 Tutorial
2 Factorizations of MultiDimensional Arrays Normally: factorize the data into a lower dimensional space in order to describe the original data in a concise manner. Focus of Lecture: Factorization of empirical joint distribution Factorization of symmetric forms Latent Class Model Probabilistic clustering. Factorization of partiallysymmetric forms Latent clustering ICML07 Tutorial
3 Nway array decompositions A rank=1 matrix G is represented by an outerproduct of two vectors: A rank=1 tensor (nway array) of n vectors is represented by an outerproduct ICML07 Tutorial
4 Nway array decompositions A matrix G is of (at most) rank=k if it can be represented by a sum of k rank1 matrices: A tensor G is (at most) rank=k if it can be represented by a sum of k rank1 tensors: Example: ICML07 Tutorial
5 Nway array Symmetric Decompositions A symmetric rank=1 matrix G: A symmetric rank=k matrix G: A supersymmetric rank=1 tensor (nway array), is represented by an outerproduct of n copies of a single vector A supersymmetric tensor described as sum of k supersymmetric rank=1 tensors: is (at most) rank=k. ICML07 Tutorial
6 General Tensors Latent Class Models ICML07 Tutorial 6
7 Reduced Rank in Statistics Let and be two random variables taking values in the sets The statement is independent of, denoted by means: is a 2D array (a matrix) is a 1D array (a vector) is a 1D array (a vector) means that is a rank=1 matrix Hebrew University
8 Reduced Rank in Statistics Let be random variables taking values in the sets The statement means: is a nway array (a tensor) is a 1D array (a vector) whose entries means that is a rank=1 tensor Hebrew University
9 Reduced Rank in Statistics Let be three random variables taking values in the sets The conditional independence statement means: Slice is a rank=1 matrix Hebrew University
10 Reduced Rank in Statistics: Latent Class Model Let Let be random variables taking values in the sets be a hidden random variable taking values in the set The observed joint probability nway array is: A statement of the form About the nway array translates to the algebraic statement having tensorrank equal to Hebrew University
11 Reduced Rank in Statistics: Latent Class Model is a nway array is a 1D array (a vector) is a rank1 nway array is a 1D array (a vector) Hebrew University
12 Reduced Rank in Statistics: Latent Class Model for n=2: Hebrew University
13 Reduced Rank in Statistics: Latent Class Model Hebrew University
14 Reduced Rank in Statistics: Latent Class Model when loss() is the relativeentropy: then the factorization above is called plsa (Hofmann 1999) Typical Algorithm: Expectation Maximization (EM) Hebrew University
15 Reduced Rank in Statistics: Latent Class Model The ExpectationMaximization algorithm introduces auxiliary tensors and alternates among the three sets of variables: Hadamard product. reduces to repeated application of projection onto probability simplex Hebrew University 15
16 Reduced Rank in Statistics: Latent Class Model An L2 version can be realized by replacing RE in the projection operation: Hebrew University
17 Reduced Rank in Statistics: Latent Class Model Let and we wish to find which minimizes: Hebrew University
18 Reduced Rank in Statistics: Latent Class Model To find given, we need to solve a convex program: Hebrew University
19 Reduced Rank in Statistics: Latent Class Model An equivalent representation: Nonnegative Matrix Factorization (normalize columns of G) Hebrew University
20 (nonnegative) Low Rank Decompositions Measurements (nonnegative) Low Rank Decompositions The rank1 blocks tend to represent local parts of the image class Hierarchical buildup of important parts 4 factors 8 factors 12 factors 16 factors 20 factors ICML07 Tutorial
21 (nonnegative) Low Rank Decompositions Example: The swimmer Sample set (256) Nonnegative Tensor Factorization NMF ICML07 Tutorial
22 Supersymmetric Decompositions Clustering over Hypergraphs Matrices ICML07 Tutorial 22
23 Clustering data into k groups: Pairwise Affinity input points input (pairwise) affinity value interpret as the probability that and are clustered together unknown class labels ICML07 Tutorial 23
24 Clustering data into k groups: Pairwise Affinity input points k=3 clusters, this point does not belong to any cluster in a hard sense. input (pairwise) affinity value interpret as the probability that and are clustered together unknown class labels A probabilistic view: probability that belongs to the j th cluster note: What is the (algebraic) relationship between the input matrix K and the desired G? ICML07 Tutorial 24
25 Clustering data into k groups: Pairwise Affinity Assume the following conditional independence statements: ICML07 Tutorial 25
26 Probabilistic MinCut A hard assignment requires that: where are the cardinalities of the clusters Proposition: the feasible set of matrices G that satisfy are of the form: equivalent to: Mincut formulation ICML07 Tutorial 26
27 Relation to Spectral Clustering Add a balancing constraint: put together means that also, and means that n/k can be dropped. Relax the balancing constraint by replacing K with the closest doubly stochastic matrix K and ignore the nonnegativity constraint: ICML07 Tutorial 27
28 Relation to Spectral Clustering Let Then: is the closest D.S. in L1 error norm. NormalizedCuts Ratiocuts Proposition: iterating with is converges to the closest D.S. in KLdiv error measure. ICML07 Tutorial 28
29 New Normalization for Spectral Clustering where we are looking for the closest doublystochastic matrix in leastsquares sense. ICML07 Tutorial 29
30 New Normalization for Spectral Clustering to find K as above, we break this into two subproblems: use the VonNeumann successive projection lemma: ICML07 Tutorial 30
31 New Normalization for Spectral Clustering successiveprojection vs. QP solver running time for the three normalizations ICML07 Tutorial 31
32 New Normalization for Spectral Clustering UCI Datasets Cancer Datasets ICML07 Tutorial 32
33 Supersymmetric Decompositions Clustering over Hypergraphs Tensors ICML07 Tutorial 33
34 Clustering data into k groups: Beyond Pairwise Affinity A model selection problem that is determined by n1 points can be described by a factorization problem of nway array (tensor). Example: clustering m points into k lines Input: Output: the probability that same model (line). belong to the the probability that the point belongs to the r th model (line) Under the independence assumption: is a 3dimensional supersymmetric tensor of rank=4 ICML07 Tutorial
35 Clustering data into k groups: Beyond Pairwise Affinity General setting: clusters are defined by n1 dim subspaces, then for each n tuple of points we define an affinity value where is the volume defined by the ntuple. Input: the probability that belong to the same cluster Output: the probability that the point belongs to the r th cluster Assume the conditional independence: is a ndimensional supersymmetric tensor of rank=k ICML07 Tutorial
36 Clustering data into k groups: Beyond Pairwise Affinity Hyperstochastic constraint: under balancing requirement K is (scaled) hyperstochastic: Theorem: for any nonnegative supersymmetric tensor, iterating converges to a hyperstochastic tensor. ICML07 Tutorial
37 Example: multibody segmentation Model Selection 9way array, each entry contains The probability that a choice of 9tuple of points arise from the same model. Probability: ICML07 Tutorial
38 Example: visual recognition under changing illumination 4way array, each entry contains The probability that a choice of 4 images Live in a 3D subspace. Model Selection ICML07 Tutorial
39 Partiallysymmetric Decompositions Latent Clustering ICML07 Tutorial 39
40 Latent Clustering Model collection of data points context states probability that are clustered together given that small high i.e., the vector has a low entropy meaning that the contribution of the context variable to the pairwise affinity is far from uniform. pairwise affinities do not convey sufficient information. input triplewise affinities
41 Latent Clustering Model Hypergraph representation G(V,E,w) Hyperedge defined over triplets representing two data points and one context state (any order) undetermined means we have no information, i.e, we do not have access to the probability that three data points are clustered together...
42 Latent Clustering Model Cluster 1 Cluster k Hyperedge.. Probability that the context state is Associated with the cluster Supersymmetric NTF Hypergraph clustering would provide means to cluster together points whose pairwise affinities are low, but there exists a subset of context states for which is high. Multiple copies of this block All the remaining entries are undefined In that case, a cluster would be formed containing the data points and the subset of context states.
43 Latent Clustering Model Multiple copies of this block All the remaining entries are undefined
44 Latent Clustering Model Multiple copies of this block All the remaining entries are undefined
45 Latent Clustering Model: Application collection of image fragments from a large collection of images unlabeled images holding k object classes, one object per image Available info: # of obj classes k, each image contains only one instance from one (unknown) object class. Clustering Task: associate (probabilistically) fragments to object classes. Challenges: Clusters and obj classes are not the same thing. Generally, # of clusters is larger than # of obj classes. Also, the same cluster may be shared among a number of different classes. The probability that two fragments should belong to the same cluster may be low only because they appear together in a small subset of images small high
46 Images Examples 3 Classes of images: Cows Faces Cars
47 Fragments Examples
48 Examples of Representative Fragments
49 Leading Fragments Of Clusters Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6
50 Examples Of The Resulting Matrix G Rows
51 Examples Of The Resulting Matrix G Rows
52 Handling Multiple Object Classes Unlabeled set of segmented images, each containing an instance of some unknown object class from a collection of 10 classes: (1) Bottle, (2) can, (3) do not enter sign,(4) stop sign, (5) one way sign,(6) frontal view car,(7) side view car,(8) face,(9) computer mouse, (10) pedestrian dataset adapted from Torralba,Murphey and Freeman, CVPR04
53 Clustering Fragments
54 From Clusters to Classes
55 Results A location which is associated with fragments voting consistently for the same object class will have high value in the corresponding voting map. Strongest hotspots (local maximums) in the voting maps show the most probable locations of objects.
56 Results
57 Applications of NTF local features for object class recognition ICML07 Tutorial 57
58 Object Class Recognition using Filters Goal : Find a good and efficient collection of filters that captures the essence of an object class of images. Two main approaches: 1) Use a large predefined bank of filters which: Rich in variability. Efficiently convolved with an image. ViolaJones HelOr 2) Use filters as basis vectors spanning a lowdimensional vector space which best fits the training images. PCA, HOSVD holistic NMF sparse The classification algorithm selects a subset of filters which is most discriminatory. Hebrew University 58
59 Object Class Recognition using NTFFilters The optimization scheme: 1) 2) Construct a bank of filters based on Nonnegative Tensor Factorization. Consider the filters as weak learners and use AdaBoost. Bank of Filters: We perform an NTF of k factors to approximate the set of original images. We perform an NTF of k factors to approximate the set of inverse images. There are filters Every filter is a pair of original/inverse factor. We take the difference between the two convolutions (original  inverse). Original/Inverse pair for face recognition Hebrew University 59
60 Object Class Recognition using NTFFilters The optimization scheme: 1) Construct a bank of filters based on Nonnegative Tensor Factorization. 2) Consider the filters as weak learners and use AdaBoost. There are original/inverse pairs of weak learners: We ran AdaBoost to construct a classifier of 50 original/inverse NTF pairs. For comparison we ran AdaBoost on other sets of weak learners: AdaBoost classifier constructed from 50 NMFweak learners. AdaBoost classifier constructed from 50 PCAweak learners. AdaBoost classifier constructed from 200 VJweak learners. An NTFFilter contains 40 multiplications. An NMF / PCA filter contains about 400 multiplications, yet we have comparable results using the same number of filters. A ViolaJones weak learner is simpler therefore we used more weaklearners in the AdaBoost classifier. Hebrew University 60
61 Face Detection using NTFFilters We recovered 100 original factors and 100 inverse factors by preforming NTF on 500 faces of size 24x24. leading filters: NTF NMF PCA Hebrew University 61
62 Face Detection using NTFFilters ROC curve: 1) For face recognition all the methods achieve comparable performance. Training: The AdaBoost was trained on 2500 faces and 4000 nonfaces 2) Test: The AdaBoost was tested on the MIT Test Set, containing 23 images with 149 faces. Hebrew University 62
63 Face Detection using NTFFilters Results Hebrew University 63
64 Pedestrian Detection using NTFFilters Sample of the database: Hebrew University 64
65 Pedestrian Detection using NTFFilters We recovered 100 original factors and 100 inverse factors by preforming NTF on 500 pedestrians of size 10x30. leading filters: NTF NMF PCA Hebrew University 65
66 Pedestrian Detection using NTFFilters ROC curve: 1) 2) For pedestrian detection NTF achieves far better performance than the rest. Training: The AdaBoost was trained on 4000 pedestrians and 4000 nonpedestrains. Test: The AdaBoost was tested on 20 images with 103 pedestrians. Hebrew University 66
67 Summary Factorization of empirical joint distribution Factorization of symmetric forms Latent Class Model Probabilistic clustering. Factorization of partiallysymmetric forms Latent clustering Further details in Hebrew University 67
68 END ICML07 Tutorial
CSE 494 CSE/CBS 598 (Fall 2007): Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye
CSE 494 CSE/CBS 598 Fall 2007: Numerical Linear Algebra for Data Exploration Clustering Instructor: Jieping Ye 1 Introduction One important method for data compression and classification is to organize
More informationKeywords: Image Generation and Manipulation, Video Processing, Video Factorization, Face Morphing
TENSORIAL FACTORIZATION METHODS FOR MANIPULATION OF FACE VIDEOS S. Manikandan, Ranjeeth Kumar, C.V. Jawahar Center for Visual Information Technology International Institute of Information Technology, Hyderabad
More informationPredict Influencers in the Social Network
Predict Influencers in the Social Network Ruishan Liu, Yang Zhao and Liuyu Zhou Email: rliu2, yzhao2, lyzhou@stanford.edu Department of Electrical Engineering, Stanford University Abstract Given two persons
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationLearning Gaussian process models from big data. Alan Qi Purdue University Joint work with Z. Xu, F. Yan, B. Dai, and Y. Zhu
Learning Gaussian process models from big data Alan Qi Purdue University Joint work with Z. Xu, F. Yan, B. Dai, and Y. Zhu Machine learning seminar at University of Cambridge, July 4 2012 Data A lot of
More informationMaximumMargin Matrix Factorization
MaximumMargin Matrix Factorization Nathan Srebro Dept. of Computer Science University of Toronto Toronto, ON, CANADA nati@cs.toronto.edu Jason D. M. Rennie Tommi S. Jaakkola Computer Science and Artificial
More informationProbabilistic Latent Semantic Analysis (plsa)
Probabilistic Latent Semantic Analysis (plsa) SS 2008 Bayesian Networks Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uniaugsburg.de www.multimediacomputing.{de,org} References
More informationSemiSupervised Support Vector Machines and Application to Spam Filtering
SemiSupervised Support Vector Machines and Application to Spam Filtering Alexander Zien Empirical Inference Department, Bernhard Schölkopf Max Planck Institute for Biological Cybernetics ECML 2006 Discovery
More informationAPPM4720/5720: Fast algorithms for big data. Gunnar Martinsson The University of Colorado at Boulder
APPM4720/5720: Fast algorithms for big data Gunnar Martinsson The University of Colorado at Boulder Course objectives: The purpose of this course is to teach efficient algorithms for processing very large
More informationFace Recognition using Principle Component Analysis
Face Recognition using Principle Component Analysis Kyungnam Kim Department of Computer Science University of Maryland, College Park MD 20742, USA Summary This is the summary of the basic idea about PCA
More informationRegression Using Support Vector Machines: Basic Foundations
Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical
More informationCheng Soon Ong & Christfried Webers. Canberra February June 2016
c Cheng Soon Ong & Christfried Webers Research Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 31 c Part I
More informationLinear Codes. Chapter 3. 3.1 Basics
Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length
More informationCI6227: Data Mining. Lesson 11b: Ensemble Learning. Data Analytics Department, Institute for Infocomm Research, A*STAR, Singapore.
CI6227: Data Mining Lesson 11b: Ensemble Learning Sinno Jialin PAN Data Analytics Department, Institute for Infocomm Research, A*STAR, Singapore Acknowledgements: slides are adapted from the lecture notes
More informationGeometricGuided Label Propagation for Moving Object Detection
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com GeometricGuided Label Propagation for Moving Object Detection Kao, J.Y.; Tian, D.; Mansour, H.; Ortega, A.; Vetro, A. TR2016005 March 2016
More informationActive Learning with Boosting for Spam Detection
Active Learning with Boosting for Spam Detection Nikhila Arkalgud Last update: March 22, 2008 Active Learning with Boosting for Spam Detection Last update: March 22, 2008 1 / 38 Outline 1 Spam Filters
More informationLecture 11: Graphical Models for Inference
Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference  the Bayesian network and the Join tree. These two both represent the same joint probability
More information203.4770: Introduction to Machine Learning Dr. Rita Osadchy
203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 19967 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More informationMachine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
More informationOrthogonal Diagonalization of Symmetric Matrices
MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding
More informationComparison of Nonlinear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data
CMPE 59H Comparison of Nonlinear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Nonlinear
More informationClassification algorithm in Data mining: An Overview
Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department
More informationSimple and efficient online algorithms for real world applications
Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRALab,
More informationDistributed Structured Prediction for Big Data
Distributed Structured Prediction for Big Data A. G. Schwing ETH Zurich aschwing@inf.ethz.ch T. Hazan TTI Chicago M. Pollefeys ETH Zurich R. Urtasun TTI Chicago Abstract The biggest limitations of learning
More informationC19 Machine Learning
C9 Machine Learning 8 Lectures Hilary Term 25 2 Tutorial Sheets A. Zisserman Overview: Supervised classification perceptron, support vector machine, loss functions, kernels, random forests, neural networks
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More informationAn Initial Study on HighDimensional Data Visualization Through Subspace Clustering
An Initial Study on HighDimensional Data Visualization Through Subspace Clustering A. Barbosa, F. Sadlo and L. G. Nonato ICMC Universidade de São Paulo, São Carlos, Brazil IWR Heidelberg University, Heidelberg,
More informationFace Recognition using SIFT Features
Face Recognition using SIFT Features Mohamed Aly CNS186 Term Project Winter 2006 Abstract Face recognition has many important practical applications, like surveillance and access control.
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More information1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
More informationAn Effective Way to Ensemble the Clusters
An Effective Way to Ensemble the Clusters R.Saranya 1, Vincila.A 2, Anila Glory.H 3 P.G Student, Department of Computer Science Engineering, Parisutham Institute of Technology and Science, Thanjavur, Tamilnadu,
More informationStatistical machine learning, high dimension and big data
Statistical machine learning, high dimension and big data S. Gaïffas 1 14 mars 2014 1 CMAP  Ecole Polytechnique Agenda for today Divide and Conquer principle for collaborative filtering Graphical modelling,
More informationCS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.
Lecture Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott
More informationHDDVis: An Interactive Tool for High Dimensional Data Visualization
HDDVis: An Interactive Tool for High Dimensional Data Visualization Mingyue Tan Department of Computer Science University of British Columbia mtan@cs.ubc.ca ABSTRACT Current high dimensional data visualization
More informationLABEL PROPAGATION ON GRAPHS. SEMISUPERVISED LEARNING. Changsheng Liu 10302014
LABEL PROPAGATION ON GRAPHS. SEMISUPERVISED LEARNING Changsheng Liu 10302014 Agenda Semi Supervised Learning Topics in Semi Supervised Learning Label Propagation Local and global consistency Graph
More informationThe Data Mining Process
Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data
More informationMachine Learning Final Project Spam Email Filtering
Machine Learning Final Project Spam Email Filtering March 2013 Shahar Yifrah Guy Lev Table of Content 1. OVERVIEW... 3 2. DATASET... 3 2.1 SOURCE... 3 2.2 CREATION OF TRAINING AND TEST SETS... 4 2.3 FEATURE
More informationCS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #18: Dimensionality Reduc7on
CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #18: Dimensionality Reduc7on Dimensionality Reduc=on Assump=on: Data lies on or near a low d dimensional subspace Axes of this subspace
More informationMachine Learning in Computer Vision A Tutorial. Ajay Joshi, Anoop Cherian and Ravishankar Shivalingam Dept. of Computer Science, UMN
Machine Learning in Computer Vision A Tutorial Ajay Joshi, Anoop Cherian and Ravishankar Shivalingam Dept. of Computer Science, UMN Outline Introduction Supervised Learning Unsupervised Learning SemiSupervised
More informationClustering Connectionist and Statistical Language Processing
Clustering Connectionist and Statistical Language Processing Frank Keller keller@coli.unisb.de Computerlinguistik Universität des Saarlandes Clustering p.1/21 Overview clustering vs. classification supervised
More informationLinear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.
Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.
More informationLearning Tools for Big Data Analytics
Learning Tools for Big Data Analytics Georgios B. Giannakis Acknowledgments: Profs. G. Mateos and K. Slavakis NSF 1343860, 1442686, and MURIFA95501010567 Center for Advanced Signal and Image Sciences
More informationSPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING
AAS 07228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations
More informationLinear Programming. March 14, 2014
Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1
More informationNonnegative Matrix Factorization (NMF) in Semisupervised Learning Reducing Dimension and Maintaining Meaning
Nonnegative Matrix Factorization (NMF) in Semisupervised Learning Reducing Dimension and Maintaining Meaning SAMSI 10 May 2013 Outline Introduction to NMF Applications Motivations NMF as a middle step
More informationObject Recognition. Selim Aksoy. Bilkent University saksoy@cs.bilkent.edu.tr
Image Classification and Object Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Image classification Image (scene) classification is a fundamental
More informationBIG DATA PROBLEMS AND LARGESCALE OPTIMIZATION: A DISTRIBUTED ALGORITHM FOR MATRIX FACTORIZATION
BIG DATA PROBLEMS AND LARGESCALE OPTIMIZATION: A DISTRIBUTED ALGORITHM FOR MATRIX FACTORIZATION Ş. İlker Birbil Sabancı University Ali Taylan Cemgil 1, Hazal Koptagel 1, Figen Öztoprak 2, Umut Şimşekli
More informationClustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016
Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with
More informationα = u v. In other words, Orthogonal Projection
Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v
More informationMachine Learning for Data Science (CS4786) Lecture 1
Machine Learning for Data Science (CS4786) Lecture 1 TuTh 10:10 to 11:25 AM Hollister B14 Instructors : Lillian Lee and Karthik Sridharan ROUGH DETAILS ABOUT THE COURSE Diagnostic assignment 0 is out:
More informationReducing multiclass to binary by coupling probability estimates
Reducing multiclass to inary y coupling proaility estimates Bianca Zadrozny Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 920930114 zadrozny@cs.ucsd.edu
More informationFUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM
International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 3448 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT
More informationNeural Networks Lesson 5  Cluster Analysis
Neural Networks Lesson 5  Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt.  Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29
More informationOverview. Clustering. Clustering vs. Classification. Supervised vs. Unsupervised Learning. Connectionist and Statistical Language Processing
Overview Clustering Connectionist and Statistical Language Processing Frank Keller keller@coli.unisb.de Computerlinguistik Universität des Saarlandes clustering vs. classification supervised vs. unsupervised
More informationLinear Algebra Review. Vectors
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length
More informationMethodology for Emulating Self Organizing Maps for Visualization of Large Datasets
Methodology for Emulating Self Organizing Maps for Visualization of Large Datasets Macario O. Cordel II and Arnulfo P. Azcarraga College of Computer Studies *Corresponding Author: macario.cordel@dlsu.edu.ph
More informationAn Overview of Knowledge Discovery Database and Data mining Techniques
An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More informationMALLETPrivacy Preserving Influencer Mining in Social Media Networks via Hypergraph
MALLETPrivacy Preserving Influencer Mining in Social Media Networks via Hypergraph Janani K 1, Narmatha S 2 Assistant Professor, Department of Computer Science and Engineering, Sri Shakthi Institute of
More informationCS 688 Pattern Recognition Lecture 4. Linear Models for Classification
CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(
More informationThese axioms must hold for all vectors ū, v, and w in V and all scalars c and d.
DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms
More informationChapter 6. The stacking ensemble approach
82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described
More informationDissertation TOPIC MODELS FOR IMAGE RETRIEVAL ON LARGESCALE DATABASES. Eva Hörster
Dissertation TOPIC MODELS FOR IMAGE RETRIEVAL ON LARGESCALE DATABASES Eva Hörster Department of Computer Science University of Augsburg Adviser: Readers: Prof. Dr. Rainer Lienhart Prof. Dr. Rainer Lienhart
More informationObject class recognition using unsupervised scaleinvariant learning
Object class recognition using unsupervised scaleinvariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of Technology Goal Recognition of object categories
More informationLocal features and matching. Image classification & object localization
Overview Instance level search Local features and matching Efficient visual recognition Image classification & object localization Category recognition Image classification: assigning a class label to
More informationMS1b Statistical Data Mining
MS1b Statistical Data Mining Yee Whye Teh Department of Statistics Oxford http://www.stats.ox.ac.uk/~teh/datamining.html Outline Administrivia and Introduction Course Structure Syllabus Introduction to
More informationSYMMETRIC EIGENFACES MILI I. SHAH
SYMMETRIC EIGENFACES MILI I. SHAH Abstract. Over the years, mathematicians and computer scientists have produced an extensive body of work in the area of facial analysis. Several facial analysis algorithms
More informationKMeans Clustering Tutorial
KMeans Clustering Tutorial By Kardi Teknomo,PhD Preferable reference for this tutorial is Teknomo, Kardi. KMeans Clustering Tutorials. http:\\people.revoledu.com\kardi\ tutorial\kmean\ Last Update: July
More informationSupport Vector Machines with Clustering for Training with Very Large Datasets
Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano
More informationSoft Clustering with Projections: PCA, ICA, and Laplacian
1 Soft Clustering with Projections: PCA, ICA, and Laplacian David Gleich and Leonid Zhukov Abstract In this paper we present a comparison of three projection methods that use the eigenvectors of a matrix
More information1. Classification problems
Neural and Evolutionary Computing. Lab 1: Classification problems Machine Learning test data repository Weka data mining platform Introduction Scilab 1. Classification problems The main aim of a classification
More informationEnsemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 20150305
Ensemble Methods Knowledge Discovery and Data Mining 2 (VU) (707004) Roman Kern KTI, TU Graz 20150305 Roman Kern (KTI, TU Graz) Ensemble Methods 20150305 1 / 38 Outline 1 Introduction 2 Classification
More informationData Mining Project Report. Document Clustering. Meryem UzunPer
Data Mining Project Report Document Clustering Meryem UzunPer 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. Kmeans algorithm...
More informationProgramming Exercise 3: Multiclass Classification and Neural Networks
Programming Exercise 3: Multiclass Classification and Neural Networks Machine Learning November 4, 2011 Introduction In this exercise, you will implement onevsall logistic regression and neural networks
More informationRecall that two vectors in are perpendicular or orthogonal provided that their dot
Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal
More informationEECS 445: Introduction to Machine Learning Winter 2015
Instructor: Prof. Jenna Wiens Office: 3609 BBB wiensj@umich.edu EECS 445: Introduction to Machine Learning Winter 2015 Graduate Student Instructor: Srayan Datta Office: 3349 North Quad (**office hours
More informationData Mining. Nonlinear Classification
Data Mining Unit # 6 Sajjad Haider Fall 2014 1 Nonlinear Classification Classes may not be separable by a linear boundary Suppose we randomly generate a data set as follows: X has range between 0 to 15
More informationClustering in Machine Learning. By: Ibrar Hussain Student ID:
Clustering in Machine Learning By: Ibrar Hussain Student ID: 11021083 Presentation An Overview Introduction Definition Types of Learning Clustering in Machine Learning Kmeans Clustering Example of kmeans
More informationNOTES ON LINEAR TRANSFORMATIONS
NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all
More information2.1: MATRIX OPERATIONS
.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and
More informationEquilibrium computation: Part 1
Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium
More informationARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING)
ARTIFICIAL INTELLIGENCE (CSCU9YE) LECTURE 6: MACHINE LEARNING 2: UNSUPERVISED LEARNING (CLUSTERING) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Preliminaries Classification and Clustering Applications
More informationLecture Topic: LowRank Approximations
Lecture Topic: LowRank Approximations LowRank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original
More informationLearning with Local and Global Consistency
Learning with Local and Global Consistency Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 7276 Tuebingen, Germany
More informationLearning with Local and Global Consistency
Learning with Local and Global Consistency Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 7276 Tuebingen, Germany
More informationA Learning Based Method for SuperResolution of Low Resolution Images
A Learning Based Method for SuperResolution of Low Resolution Images Emre Ugur June 1, 2004 emre.ugur@ceng.metu.edu.tr Abstract The main objective of this project is the study of a learning based method
More informationLearning from Diversity
Learning from Diversity Epitope Prediction with Sequence and Structure Features using an Ensemble of Support Vector Machines Rob Patro and Carl Kingsford Center for Bioinformatics and Computational Biology
More informationCrowdclustering with Sparse Pairwise Labels: A Matrix Completion Approach
Outline Crowdclustering with Sparse Pairwise Labels: A Matrix Completion Approach Jinfeng Yi, Rong Jin, Anil K. Jain, Shaili Jain 2012 Presented By : KHALID ALKOBAYER Crowdsourcing and Crowdclustering
More informationCS 534: Computer Vision 3D Modelbased recognition
CS 534: Computer Vision 3D Modelbased recognition Ahmed Elgammal Dept of Computer Science CS 534 3D Modelbased Vision  1 High Level Vision Object Recognition: What it means? Two main recognition tasks:!
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationLargeScale Similarity and Distance Metric Learning
LargeScale Similarity and Distance Metric Learning Aurélien Bellet Télécom ParisTech Joint work with K. Liu, Y. Shi and F. Sha (USC), S. Clémençon and I. Colin (Télécom ParisTech) Séminaire Criteo March
More informationGuido Sciavicco. 11 Novembre 2015
classical and new techniques Università degli Studi di Ferrara 11 Novembre 2015 in collaboration with dr. Enrico Marzano, CIO Gap srl Active Contact System Project 1/27 Contents What is? Embedded Wrapper
More informationA Network Flow Approach in Cloud Computing
1 A Network Flow Approach in Cloud Computing Soheil Feizi, Amy Zhang, Muriel Médard RLE at MIT Abstract In this paper, by using network flow principles, we propose algorithms to address various challenges
More informationIntroduction to Machine Learning Using Python. Vikram Kamath
Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression
More informationRobotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard
Robotics 2 Clustering & EM Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard 1 Clustering (1) Common technique for statistical data analysis to detect structure (machine learning,
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More informationComplex Networks Analysis: Clustering Methods
Complex Networks Analysis: Clustering Methods Nikolai Nefedov Spring 2013 ISI ETH Zurich nefedov@isi.ee.ethz.ch 1 Outline Purpose to give an overview of modern graphclustering methods and their applications
More informationBig graphs: Theory and Practice, January 68, 2016, UC San Diego. Abstracts
Big graphs: Theory and Practice, January 68, 2016, UC San Diego Anima Anandkumar (UC Irvine) Abstracts Learning mixed membership community models via spectral methods Abstract: Learning hidden communities
More information