Machine Learning. 01 - Introduction

Machine Learning 01 - Introduction

Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge is required: linear algebra, differentiation, statistics, optimization... Exercises will be done on computers, by programming in Python or using the algorithms seen in the course. The first exercise is next Monday 19.10, with a tutorial on Python. Please send me an email so that I can send you infos (illness, change of room, etc).

Some definitions of Machine Learning Arthur Samuel (1959) Machine learning: field of study that gives computers the ability to learn without being explicitely programmed. Herbert Simon (1983) Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more efficiently and more effectively the next time. Tom Mitchell (1997) A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

Automated learning is required when: Human expertise does not exist (navigating on Mars, genome sequencing). Humans are unable to explain their expertise (speech recognition). The environment constantly changes with time (routing on a computer network, realistic robot). The solution needs to be adapted to particular cases (biometrics). The data to be analysed is too big (data-mining, web-search).

Machine learning as an interdisciplinary field Machine learning systems learns general models from a data set of particular examples. Artificial intelligence: decision trees, inductive logic programming, problem solving, explanation-based learning... Statistics: how to use samples drawn from unknown distributions to help decide from which distribution some new sample is drawn. Computational neuroscience: machine learning algorithms are often related to artificial neural networks, and somehow to biological neural networks. Psychology: machine learning paradigms are inspired from psychological learning concepts, such as Pavlovian conditioning, reinforcement learning... Evolution: genetic algorithms on computers allow to confound learning and evolving.

Applications of machine learning on big databases Web-mining: search engines, online translation Showing results for machine learning. Search instead for machine laerning. Retail: market analysis, customer relationship analysis People who bought Harry Potter also bought Magic for dummies. Social networks: suggestions, optimization of resources People you also may know: Vladimir Vapnik. Finance: credit scoring, fraud detection Medicine: medical diagnosis, imaging Bioinformatics: motifs, alignement, genome sequencing Pattern recognition: face, voice, text, fingerprint, gesture, brain waves

Classification of ML techniques: Supervised machine learning: The program is "trained" on a pre-defined set of "training examples", which then facilitate its ability to reach an accurate conclusion when given new data. Unsupervised machine learning: The program is given a bunch of data and must find patterns and relationships therein. Reinforcement learning: The program explores its environment by producing actions and receiving rewards. But also: Semi-supervised learning, self-taught learning, developmental learning...

Supervised learning : regression

Supervised learning : classification

http://demo.caffe.berkeleyvision.org Applications of supervised learning : object recognition

Picasa, Facebook... Applications of supervised learning : facial recognition

Applications of supervised learning : handwriting recognition The MNIST database is the simplest benchmark for object recognition (> 99.5 %) Automatic recognition of ZIP code: SVM, convolutional networks

Applications of supervised learning : handwriting recognition 0:48

Applications of supervised learning : speech recognition Siri, Google Now, Cortana... Based on deep learning, Hidden Markov models, hidden latent variables...

Applications of supervised learning : BCI Brain-Computer Interface An EEG system can record brain potentials when performing certain tasks (thinking of moving left, raising the hand, etc) This temporal data is classified by a SVM. In use, the result of the prediction is sent to an effector: robot, wheelchair, robotic hand, videogame, etc...

Applications of supervised learning : BCI Brain-Computer Interface 6:26:39

Applications of supervised learning: Facial expression recognition 0:10

Unsupervised learning In unsupervised learning, only input data is provided to the algorithm, which has to analyse the statistical properties of the data. The goal of unsupervised learning is to build a model or find useful representations of the data, for example: finding clusters of similar data and model their density dimensionality reduction finding good explanations (hidden causes) of the data

Applications of unsupervised learning : Learning topologies

Applications of unsupervised learning: Blind source separation Cocktail party problem: isolating a source from other ones. The separation is possible thanks to the statistical properties of the signals and to an hypothesis on their original distribution. This hypothesis (tone, distance) is called belief and allows to apply bayesian inference to the problem. Applications in signal processing, neural recordings, etc.

Applications of unsupervised learning: Understanding vision Unsupervised learning allows to understand self-organisation mechanisms. Neurons in the visual cortex develop receptive fields (type of visual stimuli to which they preferentially respond) that have an oriented ON-center OFF-surround shape (Gabor filters). One unsupervised learning algorithm, the independent component analysis (ICA), develops similar receptive fields when learning natural images. This algorithm relies on efficient (sparse) coding, i.e. each neuron learns a visual feature by minimizing its correlation with the other neurons.

Applications of unsupervised learning: Deep learning Self-taught learning: feature extraction is performed on huge unlabelled datasets (e.g. Youtube videos), before being applied to supervised problems. Deep learning networks can be well parallelized: performance. Current buzzword in ML research and IT (Google, Facebook...)

Reinforcement learning The agent interacts with its environment and learns a policy to map a state into an action in order to maximize the reward obtained from the environment. The correct action is never given, only if it correct or not. The system has therefore to try out all possibilities (exploration). Optimization of performance and approximation of functions. Drawback: the exploratory space can become huge.

Applications of reinforcement learning: Games Simple RL techniques back in 1995 have found the optimal strategy to play the backgammon. No previous knowledge of Backgammon: the algorithm learned the optimal strategy by playing against itself. RL algorithms are also applied to Poker, Chess or Go, but with less success until now. The main problem for RL is that reward (i.e. win or loss) should not be delayed too much, as in a game that never ends.

Applications of reinforcement learning: Games 2:34 Volodymyr et al (2015). Human-level control through deep reinforcement learning. Nature 518, 529 533. doi:10.1038/nature14236

Applications of reinforcement learning: Robotics

Applications of reinforcement learning: Robotics 6:33:03

Applications of reinforcement learning: Robotics 6:41:36

Overview of the course 1. Supervised Learning 1. Linear Classification, gradient descent, Perceptron learning rule 2. Linear Regression 3. Learning theory 4. Multi-layer perceptron 5. Radial Basis Function network 6. Support vector machines 2. Unsupervised Learning 1. K-means, clustering 2. PCA, LDA 3. Self-organization 4. Deep Learning 3. Reinforcement Learning 1. Formal definition of the RL-Problem 2. Dynamic Programming 3. Monte Carlo Methods 4. Temporal Difference Learning (TD) 5. Eligibility Traces, Generalization and Function Approximation

Literature Neural Networks and Learning Machines, Simon Haykin, Pearson International Edition. http://www.pearsonhighered.com/haykin Support Vector Machines and other kernel-based learning methods, Nello Cristianini and John Shawe- Taylor, Cambridge Press. http://www.support-vector.net Reinforcement Learning, Richard Sutton and Andrew Barto, MIT Press. http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html Pattern Recognition and Machine Learning, Christopher Bishop, Springer-Verlag. http://research.microsoft.com/en-us/um/people/cmbishop/prml CS229 Machine Learning course, Andrew Ng, Stanford University. http://cs229.stanford.edu And any MOOC...