Machine Learning with MATLAB U. M. Sundar Senior Application Engineer MathWorks India

Transcription

1 Machine Learning with MATLAB U. M. Sundar Senior Application Engineer MathWorks India 2013 The MathWorks, Inc. 1

2 Agenda Machine Learning Overview Machine Learning with MATLAB Unsupervised Learning Clustering Supervised Learning Classification Regression Learn More Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group

3 Machine Learning Overview Types of Learning, Categories of Algorithms Type of Learning Categories of Algorithms Unsupervised Learning Clustering Machine Learning Group and interpret data based only on input data Supervised Learning Develop predictive model based on both input and output data Classification Regression 3

4 Machine Learning When and where it is used? When to use it Predict a future outcome based on Historical data (many variables) Specific patterns Define a System that is Examples Based on inputs and outputs from the system complex to define using governing equations (e.g., black-box modeling) Pattern recognition (speech, images) Financial algorithms (credit scoring, algo trading) Energy forecasting (load, price) Biology (tumor detection, drug discovery) AAA 93.68% AA A BBB BB B CCC 2.44% 0.14% 0.03% 0.03% D 5.55% 92.60% 4.18% 0.23% 0.12% 0.59% 4.03% 91.02% 7.49% 0.73% 0.11% 0.18% 0.73% 3.90% 87.86% 8.27% 0.82% 0.37% 0.15% 0.60% 3.78% 86.74% 9.64% 1.84% 0.08% 0.39% 3.28% 85.37% 6.24% 0.06% 0.18% 2.41% 81.88% 0.06% 0.08% 0.16% 0.64% 1.64% 9.67% 10 AAA AA A BBB BB B CCC D 4

5 Basic Concepts in Machine Learning Start with an initial set of data Learn from this data Train your algorithm with this data Use the resulting model to predict outcomes for new data sets Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group

6 Machine Learning Process Exploration Modeling Evaluation Deployment 6

7 Exploratory Data Analysis Gain insight from visual examination Identify trends and interactions Detect patterns Remove outliers Shrink data Select and pare predictors Feature transformation Acceleration MPG Displacement Weight Horsepower MPG Acceleration Displacement Weight Horsepow er 7

8 Data Exploration Interactions Between Variables Acceleration MPG Weight Displacement Coordinate Value f(t) Horsepower MPG Acceleration Displacement Weight Horsepow er -2-3 MPG Acceleration Displacement Weight Horsepower t Plot Matrix by Group Parallel Coordinates Plot Andrews Plot chevrolet chevelle malibu buick skylark 320 plymouth satellite chevrolet chevelle malibuick skylark 320 plymouth satellite amc rebel sst ford torino ford galaxie 500 amc rebel sst ford torino ford galaxie 500 chevrolet impala plymouth fury iii pontiac catalina chevrolet impala plymouth fury iii pontiac catalina Glyph Plot Chernoff Faces 8

9 Unsupervised Learning Clustering K-means, Fuzzy K-means Unsupervised Learning Clustering Hierarchical Neural Network Machine Learning Group and interpret data based only on input data Supervised Learning Classification Regression Gaussian Mixture 9

10 Dataset We ll Be Using Cloud of randomly generated points Each cluster center is randomly chosen inside specified bounds Each cluster contains the specified number of points per cluster Each cluster point is sampled from a Gaussian distribution Multi-dimensional dataset Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group

11 Clustering Overview What is clustering? Segment data into groups, based on data similarity Why use clustering? Identify outliers Resulting groups may be the matter of interest How is clustering done? Can be achieved by various algorithms It is an iterative process (involving trial and error)

12 Clustering: K Means Clustering Statistics Toolbox K-means is a partitioning method Partitions data into k mutually exclusive clusters Each cluster has a centroid (or center) Sum of distances from all objects to the center is minimized 12

13 Clustering: Neural Networks Networks are comprised of one or more layers Outputs computed by applying a nonlinear transfer function with weighted sum of inputs Input variables Weights Transfer function Output Variable Trained by letting the network continually adjust itself to new inputs (determines weights) Bias Interactive apps for easily creating and training networks Multi-layered networks created by cascading (provide better accuracy) Example architectures for clustering: Self-organizing maps Competitive layers 13

14 Clustering : Gaussian Mixture Models Good when clusters have different sizes and are correlated Statistics Toolbox Assume that data is drawn from a fixed number K of normal distributions

15 Cluster Analysis Summary Segments data into groups, based on data similarity No method is perfect (depends on data) Process is iterative; explore different algorithms Clustering K-means, Fuzzy K-means Hierarchical Neural Network Beware of local minima (global optimization can help) Gaussian Mixture 15

16 Model Development Process Exploration Modeling Evaluation Deployment 16

17 Supervised Learning Classification for Predictive Modeling Unsupervised Learning Machine Learning Classification Decision Tree Ensemble Method Supervised Learning Develop predictive model based on both input and output data Neural Network Support Vector Machine 17

18 Classification Overview What is classification? Predicting the best group for each point Learns from labeled observations Uses input features Why use classification? Accurately group data never seen before Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group How is classification done? Can use several algorithms to build a predictive model Good training data is critical 18

19 Classification - Decision Trees Statistics Toolbox Builds a tree from training data Model is a tree where each node is a logical test on a predictor Traverse tree by comparing features with threshold values The leaf of the tree specifies the group 19

20 Classification - Ensemble Learners Statistics Toolbox Overview 1.5 Decision trees are weak learners Good to classify data used to train Often not very good with new data Note rectangular groups group2 group3 1 group4 group5 group6 x2 group1 group7 0.5 group8 0 What are ensemble learners? x1 Combine many decision trees to create a strong learner Uses bootstrapped aggregation Why use ensemble methods? Classifier has better predictive power Note improvement in cluster shapes 20

21 Classification - Support Vector Machines Overview Good for modeling with complex boundaries between groups Can be very accurate No restrictions on the predictors Statistics Toolbox (as of R2013a) 1 2 Support Vectors What does it do? Uses non-linear kernel to calculate the boundaries Can be computationally intensive Version in Statistics Toolbox only classifies into two groups 21

22 K-Nearest Neighbor Classification One of the simplest classifiers Takes the K nearest points from the training set, and chooses the majority class of those K points No training phase all the work is done during the application of the model 1.5 group1 group2 group3 1 group4 group5 group6 x2 Statistics Toolbox group7 0.5 group x1 22

23 Classification Summary No absolute best method Simple does not mean inefficient Classification Decision Tree Ensemble Method Neural Network Watch for overfitting Decision trees and neural networks may overfit the noise Use ensemble learning and cross-validation Support Vector Machine Parallelize for speedup 23

24 Supervised Learning Regression for Predictive Modeling Unsupervised Learning Machine Learning Supervised Learning Develop predictive model based on both input and output data Regression Linear Non-linear Non-parametric 24

25 Regression Statistics Toolbox Curve Fitting Toolbox Why use regression? Predict the continuous response for new observations Type of predictive modeling Specify a model that describes Y as a function of X Estimate coefficients that minimize the difference between predicted and actual You can apply techniques from earlier sections with regression as well 25

26 Linear Regression Y is a linear function of the regression coefficients Common examples: Straight line Y = B 0 + B 1 X 1 Plane Y = B 0 + B 1 X 1 +B 2 X 2 Polynomial Y = B 0 + B 1 X 13 + B 2 X 1 2 +B 3 X 1 Polynomial with cross terms Y = B 0 + B 1 X 12 + B 2 (X 1 X 2 ) + B 3 X

27 Nonlinear Regression Y is a nonlinear function of the regression coefficients Syntax for formulas: Fourier Series y ~ b0 + b1*cos(x*b3) + b 0 + b 1 cos b 3 X + b 2 sin b 3 X b4*sin(x*b3) Exponential Growth = N 0 + exp(- Logistic Growth P t = b(2)*x))) 1 + b 1 e kt b 0 27

28 Generalized Linear Models Extends the linear model Define relationship between model and response variable Model error distributions other than normal Logistic regression Response variable is binary (true / false) Results are typically expressed as an odd s ratio Poisson regression Model count data (non-negative integers) Response variable comes from a Poisson distribution 28

29 Machine Learning with MATLAB Interactive environment Visual tools for exploratory data analysis Easy to evaluate and choose best algorithm Apps available to help you get started (e.g,. neural network tool, curve fitting tool) Multiple algorithms to choose from Clustering Classification Regression 29

30 Learn More : Machine Learning with MATLAB machine-learning.html Data Driven Fitting with MATLAB Classification with MATLAB Regression with MATLAB Multivariate Classification in the Life Sciences Electricity Load and Price Forecasting Credit Risk Modeling with MATLAB 30

31 MathWorks India Services and Offerings Consulting Local website: Technical Support India: myservicerequests Product Training: Customer Service for non-technical questions: Application Engineering 31

32 Scheduled Public Training for Sep Dec 2013 Course Name Location Training dates Statistical Methods in MATLAB Bangalore Sep 2013 MATLAB based Optimization Techniques Bangalore 04 Sep 2013 Physical Modeling of Multi-Domain Systems using Simscape Bangalore 05 Sep 2013 Delhi Sep 2013 Pune Oct 2013 MATLAB Fundamentals Bangalore Oct 2013 Web based Nov 2013 Chennai Dec 2013 Delhi Sep 2013 Pune Oct 2013 Simulink for System and Algorithm Modeling Bangalore Oct 2013 Web based Nov 2013 Chennai Dec 2013 MATLAB Programming Techniques Bangalore Nov 2013 MATLAB for Data Processing and Visualization Bangalore 20 Nov 2013 MATLAB for Building Graphical User Interface Bangalore 21 Nov 2013 Generating HDL Code from Simulink Bangalore Nov URL: Phone:

33 MathWorks Certification Program- for the first time in India! MathWorks Certified MATLAB Associate Exam Why certification? Validates proficiency with MATLAB Can help accelerate professional growth Can help increase productivity and project success and thereby prove to be a strategic investment Certification exam administered in English at MathWorks facilities in Bangalore on Nov 27, training@mathworks.in URL: Phone:

34 MathWorks India Contact Details MathWorks India Private Limited Salarpuria Windsor Building Third Floor, No.3 Ulsoor Road Bangalore , Karnataka India URL: info@mathworks.in Technical Support: Tel: Fax: Thank You for Attending Talk to Us We are Happy to Support You 34