9.54 class 2 Types of learning + biophysical mechanisms of plasticity Shimon Ullman + Tomaso Poggio Danny Harari + Daniel Zysman + Darren Seibert 9.54, fall semester 24
An introduction to basic Machine Learning! oncepts Algorithms 9.54, fall semester 24
oncepts and algorithms in Machine Learning References: T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Prediction, Inference and Data Mining. Second Edition, Springer Verlag, 29 (available for free from the author's website). Further readings : T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 23 Pedro Domingos. A few useful things to know about machine learning. ommunications of the AM AM Homepage archive. Volume 55 Issue, October 22 Pages 78-87..! Useful Links MIT 9.52: Statistical Learning Theory and Applications, Fall 23 (http://www.mit.edu/~9.52/). Stanford S229 Machine Learning Autumn 23 (http://cs229.stanford.edu). See also the oursera version (https://www.coursera.org/course/ml).! 9.54, fall semester 24
4
5
~2 years ago in my group 6
~7 years ago in my group 7
8
9
Levels of Understanding (977--22)! Evolution! Learning and development! omputation! Algorithms! Wetware, hardware, circuits and components
uild intelligent machines Understand Learning Learning is the gateway to understanding intelligence
(Artifical) Intelligence: A Working Definition Turing test Ingredients for AI! natural language processing! knowledge representation! automated reasoning! machine learning! computer vision! robotics to manipulate
Exciting Days for AI Artificial intelligence systems have recently seen some striking successes Learning
ML and AI Machine Learning systems are trained on examples rather than being programmed
Data Unsupervised Feature Learning lassic Supervised Learning Learning Structure.. Dynamic Learning Machine Learning Problems and Approaches!
asic Setting: lassification (x,y ),...,(x n,y n ) x i 2 R p and y i 2 Y = {, }, i =,...,n X n = x......... x p..... x n......... x p n A Y n = y. y n A
Genomics n patients p gene expression measurements... ; X n = x......... x p..... x n......... x p n A; Y n = y. y n A...
Text lassification: ag of Words X n = x......... x p..... x n......... x p n A
Image lassification......
Image lassification X n = x......... x p..... x n......... x p n A
Video lassification: Action Recognition
From classification to regression (x,y ),...,(x n,y n ) x i 2 R D and y i 2 Y = {, }, i =,...,n y i 2 Y 2 R, i =,...,n housing prices Living area (feet 2 ) Price ($s) 24 4 6 33 24 369 46 232 3 54.. price (in $) 9 8 7 6 5 4 3 2 Living area (feet 2 ) #bedrooms Price ($s) 24 3 4 6 3 33 24 3 369 46 2 232 3 4 54... 5 5 2 25 3 35 4 45 5 square feet y i = f(x i )+ " i, > e.g. f(x) =w T x, " i N(, )
! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches
Variations on a Theme (x,y ),...,(x n,y n ) Multiclass: x i 2 R D and y i 2 Y = {,...,T}, i =,...,n Multitask: x i 2 R D and y i 2 R T, i =,...,n Learning a similarity function (x,x ; y, ), (x,x 2 ; y,2 )...,(x n,x n ; y n,n ) x j,x i 2 R D and y i,j 2 [, ], j, i =,...,n
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A
Semisupervised Learning
Semisupervised Learning n = x......... x p..... x n......... x p n A Y n = ; y. y n A [ X u = x......... x p..... x u......... x p u A Manifold Learning
Unsupervised Learning X n = x......... x p..... x n......... x p n A Y n = y. y n A Given x,...,x n Goal: Extract patterns... lustering, k-means! Vector Quantization! Dimensionality reduction!...
! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...!! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches
! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches
Online/Incremental Learning (x,y ),...,(x n,y n ) f f (x,y ) (x 2,y 2 ) f 2... (x n,y n ) f n (x,y ) f...
The final estimator is constructed assembling the estimate away from th boundary obtained in the preview step with the estimate in the vicinity of th f Active Learning Learner can query points (x,y ) f (x 2,y 2 ) f 2... (a) (b) (x n,y n ) f n... (c) (d) Figure 8.7. The two step procedure for d =2:(a)InitialunprunedRDPandn/2 sample (b) Preview step RDP. Note that the cell with the arrow was pruned, but it contains a part of th boundary. (c) Additional sampling for the refinement step. (d) Refinement step.
Summary ``Learning is the acquisition of knowledge or skills through study, experience, or being taught We look for systems that are trained, rather than programmed, to perform a task! Learning from examples is a unifying paradigm in learning:!