Machine Learning Introduction

Transcription

1 Machine Learning Introduction Ingmar Schuster Patrick Jähnichen Institut für Informatik

2 This lecture covers Machine Learning Overview Example applications of Machine Learning Distinction Supervised vs. Unsupervised Beyond Supervised / Unsupervised Distinction Generative vs. Discriminative Models Basic Probability Theory 2

3 Machine Learning: foundations and definitions Definitions [Giving] computers the ability to learn without being explicitly programmed - Arthur Samuel A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E - Tom M. Mitchell Common foundations Probability Theory Decision Theory (making good decisions based on data) Optimization (optimize model of data based on decision objective) independent of application domain 3

4 Applications of Machine Learning 4

5 Application: Panorama Stitching (using Markov Random Fields) 5

6 Application: Brain Computer Interface (classification algorithm) [youtu.be/jxpjrwpqc5q] 6

7 Application: Self-driving car [youtu.be/cdgqpa1puue] 7

8 Application Domains of Machine Learning Computer Vision Gene Analysis Network security Textmining & NLP: every domain Parsing POS-Tagging, Typing prediction Modelling Language Semantics Topic Models 8

9 Supervised vs. Unsupervised 9

10 Supervised vs. unsupervised learning General distinction: Supervised vs. Unsupervised Learning Supervised Learning (given expected answer, build prediction model) Classification e.g. training data contains tumor size, growth rate & diagnosis Regression e.g. Training data contains review text & product rating Rating: Rating: 88 (out (out of of 10) 10) II really really liked liked the the picture. picture. The The story story of of Johns Johns struggle struggle against against the the gremlins gremlins touched touched my my heart. heart. 10

11 Unsupervised learning Unsupervised learning (find structure in data) Clustering Find which datapoints relate to one another [Blei, Ng & Jordan 2003] 11

12 Supervised vs. unsupervised learning Unsupervised learning (find structure in data) Anomaly Detection Find datapoints that strongly deviate from the majority Possible Possible attacks attacks Regular Regular network network traffic traffic 12

13 Supervised vs. unsupervised learning Unsupervised learning (find structure in data) Dimensionality Reduction, Source Separation Find most important sources of variance in data [cnl.salk.edu/~tewon/blind/blind_audio.html] 13

14 Beyond Supervised / Unsupervised 14

15 Beyond supervised/unsupervised (1) Active learning (child constantly asking) In between supervised and unsupervised Algorithm asks for labels of only most important data points 15

16 Beyond supervised/unsupervised (2) Reinforcement learning (training a dog) Algorithm is told when it is right, but not why 16

17 Beyond supervised/unsupervised (3) Semi-Supervised learning & Transfer learning (you after college) In between supervised and unsupervised Some data is labeled, but most of it isn't 17

18 Beyond supervised/unsupervised (4) and all kinds of remixes. 18

19 Generative vs. Discriminative 19

20 Discriminative Models Discriminative Models Smokes Good if used for only one specific prediction task Models target distribution directly Doesn't model datagenerating process Cancer Xray diagnosis Breathing difficulty Can't produce synthetic data samples Good prediction performance often sufficient 20

21 Generative Models Generative models Smokes Models data-generating process in real world Enables generation of synthetic data Models joint distribution of random variables (often through distinct conditional probabilities) Cancer Xray diagnosis Breathing difficulty Very flexible 21

22 Basic Probability Theory 22

23 Random variables can take different values, written in uppercase letters ( ) Values (events) for random variables written in lowercase letters ( ) Events can be Continuous or Discrete NameType p(x) person 0.4 location 0.3 other 0.3 can be Numbers named values Trees any other type of object 23

24 Probability of an event or Distribution of random variable is function mapping event to probability NameType p(x) person 0.4 location 0.3 other

25 Joint probability of two events Conditional probability or Smokes Cancer p(s,c) Smokes Cancer p(c s) yes yes 0.1 yes yes 0.25 no yes 0.06 no yes 0.1 yes no 0.3 yes no 0.75 no no 0.54 no no

26 This lecture covered Probabilites sum to 1 Marginalization Summing/Integrating out random variable S C p(s,c) yes yes 0.1 no yes 0.06 yes no 0.3 no no

27 This lecture covered Machine Learning Overview Example applications of Machine Learning Distinction Supervised vs. Unsupervised Beyond Supervised / Unsupervised Distinction Generative vs. Discriminative Models Basic Probability Theory 27

28 Pictures Tumor picture by flickr-user bc the path, License CC SA NC Other pictures from openclipart.org, public domain 28