Machine Learning. Lecture Slides for. ETHEM ALPAYDIN The MIT Press, 2010 Edited and expanded for CS 4641 by Chris Simpkins

Transcription

1 Lecture Slides for INTRODUCTION TO Machine Learning ETHEM ALPAYDIN The MIT Press, 2010 Edited and expanded for CS 4641 by Chris Simpkins h1p:// 1

2 CHAPTER 1: IntroducOon 2

3 Why Learn? Machine learning is programming computers to opomize a performance criterion using example data or past experience. There is no need to learn to calculate payroll Learning is used when: Human experose does not exist (navigaong on Mars), Humans are unable to explain their experose (speech recognioon) SoluOon changes in Ome (rouong on a computer network) SoluOon needs to be adapted to parocular cases (user biometrics) 3 3

4 What We Talk About When We Talk About Learning Learning general models from a data of parocular examples Data is cheap and abundant (data warehouses, data marts); knowledge is expensive and scarce. Example in retail: Customer transacoons to consumer behavior: People who bought Blink also bought Outliers ( Build a model that is a good and useful approximagon to the data. 4 4

5 Data Mining Retail: Market basket analysis, Customer relaoonship management (CRM) Finance: Credit scoring, fraud detecoon Manufacturing: Control, roboocs, troubleshooong Medicine: Medical diagnosis TelecommunicaOons: Spam filters, intrusion detecoon BioinformaOcs: MoOfs, alignment Web mining: Search engines

6 What is Machine Learning? OpOmize a performance criterion using example data or past experience. Role of StaOsOcs: Inference from a sample Role of Computer science: Efficient algorithms to Solve the opomizaoon problem RepresenOng and evaluaong the model for inference 6 6

7 Types of Machine Learning AssociaOon Supervised Learning ClassificaOon Regression Unsupervised Learning Reinforcement Learning 7 7

8 Learning AssociaOons Basket analysis: P (Y X ) probability that somebody who buys X also buys Y where X and Y are products/services. Example: P ( chips beer ) =

9 ClassificaOon Example: Credit scoring DifferenOaOng between low- risk and high- risk customers from their income and savings Discriminant: IF income > θ 1 AND savings > θ 2 THEN low- risk ELSE high- risk 9 9

10 ClassificaOon: ApplicaOons Aka Pamern recognioon Face recognioon: Pose, lighong, occlusion (glasses, beard), make- up, hair style Character recognioon: Different handwriong styles. Speech recognioon: Temporal dependency. Medical diagnosis: From symptoms to illnesses Biometrics: RecogniOon/authenOcaOon using physical and/or behavioral characterisocs: Face, iris, signature, etc

11 Face RecogniOon Training examples of a person Test images ORL dataset, AT&T Laboratories, Cambridge UK 11 11

12 Regression Example: Price of a used car x : car amributes y : price y = g (x θ ) g ( ) model, θ parameters y = wx+w

13 Regression ApplicaOons NavigaOng a car: Angle of the steering KinemaOcs of a robot arm (x,y) α 2 α 1 = g 1 (x,y) α 2 = g 2 (x,y) α 1 n Response surface design 13 13

14 Supervised Learning: Uses PredicOon of future cases: Use the rule to predict the output for future inputs Knowledge extracoon: The rule is easy to understand Compression: The rule is simpler than the data it explains Outlier detecoon: ExcepOons that are not covered by the rule, e.g., fraud 14 14

15 Unsupervised Learning Learning what normally happens No output Clustering: Grouping similar instances Example applicaoons Customer segmentaoon in CRM Image compression: Color quanozaoon BioinformaOcs: Learning moofs 15 15

16 Reinforcement Learning Learning a policy: A sequence of outputs No supervised output but delayed reward Credit assignment problem Game playing Robot in a maze MulOple agents, paroal observability,

17 Resources: Datasets UCI Repository: hmp:// UCI KDD Archive: hmp://kdd.ics.uci.edu/ summary.data.applicaoon.html Statlib: hmp://lib.stat.cmu.edu/ Delve: hmp://

18 Resources: Journals Journal of Machine Learning Research Machine Learning Neural ComputaOon Neural Networks IEEE TransacOons on Neural Networks IEEE TransacOons on Pamern Analysis and Machine Intelligence Annals of StaOsOcs Journal of the American StaOsOcal AssociaOon

19 Resources: Conferences InternaOonal Conference on Machine Learning (ICML) European Conference on Machine Learning (ECML) Neural InformaOon Processing Systems (NIPS) Uncertainty in ArOficial Intelligence (UAI) ComputaOonal Learning Theory (COLT) InternaOonal Conference on ArOficial Neural Networks (ICANN) InternaOonal Conference on AI & StaOsOcs (AISTATS) InternaOonal Conference on Pamern RecogniOon (ICPR)

20 Resources: Georgia Tech h"p://ml.cc.gatech.edu/ Charles Isbell (my advisor) Reinforcement Learning Andrea Thomaz Social Learning Alex Gray StaFsFcal Learning Theory, ML SoIware Irfan Essa AcFvity RecogniFon James Rehg Graphical Models Maria Balcan Machine Learning Theory Frank Dellaert Computer Vision 20 20