Machine Learning with MATLAB U. M. Sundar Senior Application Engineer MathWorks India 2013 The MathWorks, Inc. 1
Agenda Machine Learning Overview Machine Learning with MATLAB Unsupervised Learning Clustering Supervised Learning Classification Regression Learn More 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-0.1 0 0.1 0.2 1 0.3 0.4 0.5 0.6 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group 8 0-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 2
Machine Learning Overview Types of Learning, Categories of Algorithms Type of Learning Categories of Algorithms Unsupervised Learning Clustering Machine Learning Group and interpret data based only on input data Supervised Learning Develop predictive model based on both input and output data Classification Regression 3
Machine Learning When and where it is used? When to use it Predict a future outcome based on Historical data (many variables) Specific patterns Define a System that is Examples Based on inputs and outputs from the system complex to define using governing equations (e.g., black-box modeling) Pattern recognition (speech, images) Financial algorithms (credit scoring, algo trading) Energy forecasting (load, price) Biology (tumor detection, drug discovery) AAA 93.68% AA A BBB BB B CCC 2.44% 0.14% 0.03% 0.03% D 5.55% 92.60% 4.18% 0.23% 0.12% 0.59% 4.03% 91.02% 7.49% 0.73% 0.11% 0.18% 0.73% 3.90% 87.86% 8.27% 0.82% 0.37% 0.15% 0.60% 3.78% 86.74% 9.64% 1.84% 0.08% 0.39% 3.28% 85.37% 6.24% 0.06% 0.18% 2.41% 81.88% 0.06% 0.08% 0.16% 0.64% 1.64% 9.67% 10 AAA AA A BBB BB B CCC D 4
Basic Concepts in Machine Learning Start with an initial set of data Learn from this data Train your algorithm with this data Use the resulting model to predict outcomes for new data sets 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group 8 0.2 0.1 0-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 5
Machine Learning Process Exploration Modeling Evaluation Deployment 6
Exploratory Data Analysis Gain insight from visual examination Identify trends and interactions Detect patterns Remove outliers Shrink data Select and pare predictors Feature transformation Acceleration MPG 40 20 20 10 Displacement 400 200 Weight 4000 2000 Horsepower 200 150 100 50 20 40 10 20 200 400 2000 4000 50 100150200 MPG Acceleration Displacement Weight Horsepow er 7
Data Exploration Interactions Between Variables Acceleration MPG 40 20 20 10 4 3 2 4 6 8 8 6 4 2 4 6 8 Weight Displacement 400 200 4000 2000 Coordinate Value 1 0-1 f(t) 0-2 -4 Horsepower 200 150 100 50 20 40 10 20 200 400 2000 4000 50 100150200 MPG Acceleration Displacement Weight Horsepow er -2-3 MPG Acceleration Displacement Weight Horsepower -6-8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 t Plot Matrix by Group Parallel Coordinates Plot Andrews Plot chevrolet chevelle malibu buick skylark 320 plymouth satellite chevrolet chevelle malibuick skylark 320 plymouth satellite amc rebel sst ford torino ford galaxie 500 amc rebel sst ford torino ford galaxie 500 chevrolet impala plymouth fury iii pontiac catalina chevrolet impala plymouth fury iii pontiac catalina Glyph Plot Chernoff Faces 8
Unsupervised Learning Clustering K-means, Fuzzy K-means Unsupervised Learning Clustering Hierarchical Neural Network Machine Learning Group and interpret data based only on input data Supervised Learning Classification Regression Gaussian Mixture 9
Dataset We ll Be Using Cloud of randomly generated points Each cluster center is randomly chosen inside specified bounds Each cluster contains the specified number of points per cluster Each cluster point is sampled from a Gaussian distribution Multi-dimensional dataset 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group 8-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 10
Clustering Overview What is clustering? Segment data into groups, based on data similarity 1 0.9 0.8 0.7 0.6 Why use clustering? Identify outliers Resulting groups may be the matter of interest 0.5 0.4 0.3 0.2 0.1 How is clustering done? Can be achieved by various algorithms It is an iterative process (involving trial and error) 0-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 11
Clustering: K Means Clustering Statistics Toolbox K-means is a partitioning method Partitions data into k mutually exclusive clusters Each cluster has a centroid (or center) Sum of distances from all objects to the center is minimized 12
Clustering: Neural Networks Networks are comprised of one or more layers Outputs computed by applying a nonlinear transfer function with weighted sum of inputs Input variables Weights Transfer function Output Variable Trained by letting the network continually adjust itself to new inputs (determines weights) Bias Interactive apps for easily creating and training networks Multi-layered networks created by cascading (provide better accuracy) Example architectures for clustering: Self-organizing maps Competitive layers 13
Clustering : Gaussian Mixture Models Good when clusters have different sizes and are correlated Statistics Toolbox Assume that data is drawn from a fixed number K of normal distributions 20 10 1 0 1 0.8 0.6 0.4 0.2 0 0 0.2 0.4 0.6 0.8 14
Cluster Analysis Summary Segments data into groups, based on data similarity No method is perfect (depends on data) Process is iterative; explore different algorithms Clustering K-means, Fuzzy K-means Hierarchical Neural Network Beware of local minima (global optimization can help) Gaussian Mixture 15
Model Development Process Exploration Modeling Evaluation Deployment 16
Supervised Learning Classification for Predictive Modeling Unsupervised Learning Machine Learning Classification Decision Tree Ensemble Method Supervised Learning Develop predictive model based on both input and output data Neural Network Support Vector Machine 17
Classification Overview What is classification? Predicting the best group for each point Learns from labeled observations Uses input features Why use classification? Accurately group data never seen before 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group 8-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 How is classification done? Can use several algorithms to build a predictive model Good training data is critical 18
Classification - Decision Trees Statistics Toolbox Builds a tree from training data Model is a tree where each node is a logical test on a predictor Traverse tree by comparing features with threshold values The leaf of the tree specifies the group 19
Classification - Ensemble Learners Statistics Toolbox Overview 1.5 Decision trees are weak learners Good to classify data used to train Often not very good with new data Note rectangular groups group2 group3 1 group4 group5 group6 x2 group1 group7 0.5 group8 0 What are ensemble learners? -0.5-0.4-0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 x1 Combine many decision trees to create a strong learner Uses bootstrapped aggregation Why use ensemble methods? Classifier has better predictive power Note improvement in cluster shapes 20
Classification - Support Vector Machines Overview Good for modeling with complex boundaries between groups Can be very accurate No restrictions on the predictors 4 3 2 Statistics Toolbox (as of R2013a) 1 2 Support Vectors What does it do? Uses non-linear kernel to calculate the boundaries Can be computationally intensive 1 0-1 -2-3 -2-1 0 1 2 3 Version in Statistics Toolbox only classifies into two groups 21
K-Nearest Neighbor Classification One of the simplest classifiers Takes the K nearest points from the training set, and chooses the majority class of those K points No training phase all the work is done during the application of the model 1.5 group1 group2 group3 1 group4 group5 group6 x2 Statistics Toolbox group7 0.5 group8 0-0.5-0.4-0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 x1 22
Classification Summary No absolute best method Simple does not mean inefficient Classification Decision Tree Ensemble Method Neural Network Watch for overfitting Decision trees and neural networks may overfit the noise Use ensemble learning and cross-validation Support Vector Machine Parallelize for speedup 23
Supervised Learning Regression for Predictive Modeling Unsupervised Learning Machine Learning Supervised Learning Develop predictive model based on both input and output data Regression Linear Non-linear Non-parametric 24
Regression Statistics Toolbox Curve Fitting Toolbox Why use regression? Predict the continuous response for new observations Type of predictive modeling Specify a model that describes Y as a function of X Estimate coefficients that minimize the difference between predicted and actual You can apply techniques from earlier sections with regression as well 25
Linear Regression Y is a linear function of the regression coefficients Common examples: Straight line Y = B 0 + B 1 X 1 Plane Y = B 0 + B 1 X 1 +B 2 X 2 Polynomial Y = B 0 + B 1 X 13 + B 2 X 1 2 +B 3 X 1 Polynomial with cross terms Y = B 0 + B 1 X 12 + B 2 (X 1 X 2 ) + B 3 X 2 2 26
Nonlinear Regression Y is a nonlinear function of the regression coefficients Syntax for formulas: Fourier Series y ~ b0 + b1*cos(x*b3) + b 0 + b 1 cos b 3 X + b 2 sin b 3 X b4*sin(x*b3) Exponential Growth N @(b,t)(b(1)*exp(b(2)*t) = N 0 e @(b,t)(1/(b(1) + exp(- Logistic Growth P t = b(2)*x))) 1 + b 1 e kt b 0 27
Generalized Linear Models Extends the linear model Define relationship between model and response variable Model error distributions other than normal Logistic regression Response variable is binary (true / false) Results are typically expressed as an odd s ratio Poisson regression Model count data (non-negative integers) Response variable comes from a Poisson distribution 28
Machine Learning with MATLAB Interactive environment Visual tools for exploratory data analysis Easy to evaluate and choose best algorithm Apps available to help you get started (e.g,. neural network tool, curve fitting tool) Multiple algorithms to choose from Clustering Classification Regression 29
Learn More : Machine Learning with MATLAB http://www.mathworks.com/discovery/ machine-learning.html Data Driven Fitting with MATLAB Classification with MATLAB Regression with MATLAB Multivariate Classification in the Life Sciences Electricity Load and Price Forecasting Credit Risk Modeling with MATLAB 30
MathWorks India Services and Offerings Consulting Local website: www.mathworks.in Technical Support India: www.mathworks.in/ myservicerequests Product Training: www.mathworks.in/training Customer Service for non-technical questions: info@mathworks.in Application Engineering 31
Scheduled Public Training for Sep Dec 2013 Course Name Location Training dates Statistical Methods in MATLAB Bangalore 02-03 Sep 2013 MATLAB based Optimization Techniques Bangalore 04 Sep 2013 Physical Modeling of Multi-Domain Systems using Simscape Bangalore 05 Sep 2013 Delhi 23-25 Sep 2013 Pune 07-09 Oct 2013 MATLAB Fundamentals Bangalore 21-23 Oct 2013 Web based 05-07 Nov 2013 Chennai 09-11 Dec 2013 Delhi 26-27 Sep 2013 Pune 10-11 Oct 2013 Simulink for System and Algorithm Modeling Bangalore 24-25 Oct 2013 Web based 12-13 Nov 2013 Chennai 12-13 Dec 2013 MATLAB Programming Techniques Bangalore 18-19 Nov 2013 MATLAB for Data Processing and Visualization Bangalore 20 Nov 2013 MATLAB for Building Graphical User Interface Bangalore 21 Nov 2013 Generating HDL Code from Simulink Bangalore 28-29 Nov 2013 Email: training@mathworks.in URL: http://www.mathworks.in/services/training Phone: 080-6632-6000 32
MathWorks Certification Program- for the first time in India! MathWorks Certified MATLAB Associate Exam Why certification? Validates proficiency with MATLAB Can help accelerate professional growth Can help increase productivity and project success and thereby prove to be a strategic investment Certification exam administered in English at MathWorks facilities in Bangalore on Nov 27,2013 Email: training@mathworks.in URL: http://www.mathworks.in/services/training Phone: 080-6632-6000 33
MathWorks India Contact Details MathWorks India Private Limited Salarpuria Windsor Building Third Floor, No.3 Ulsoor Road Bangalore - 560042, Karnataka India URL: http://www.mathworks.in E-mail: info@mathworks.in Technical Support: www.mathworks.in/myservicerequests Tel: +91-80-6632 6000 Fax: +91-80-6632 6010 Thank You for Attending Talk to Us We are Happy to Support You 34