9.54 class 2. Shimon Ullman + Tomaso Poggio. Types of learning + biophysical mechanisms of plasticity. Danny Harari + Daniel Zysman + Darren Seibert

Transcription

1 9.54 class 2 Types of learning + biophysical mechanisms of plasticity Shimon Ullman + Tomaso Poggio Danny Harari + Daniel Zysman + Darren Seibert 9.54, fall semester 24

2 An introduction to basic Machine Learning! oncepts Algorithms 9.54, fall semester 24

3 oncepts and algorithms in Machine Learning References: T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Prediction, Inference and Data Mining. Second Edition, Springer Verlag, 29 (available for free from the author's website). Further readings : T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 23 Pedro Domingos. A few useful things to know about machine learning. ommunications of the AM AM Homepage archive. Volume 55 Issue, October 22 Pages ! Useful Links MIT 9.52: Statistical Learning Theory and Applications, Fall 23 ( Stanford S229 Machine Learning Autumn 23 ( See also the oursera version ( 9.54, fall semester 24

4 4

5 5

6 ~2 years ago in my group 6

7 ~7 years ago in my group 7

8 8

9 9

10

11 Levels of Understanding ( )! Evolution! Learning and development! omputation! Algorithms! Wetware, hardware, circuits and components

12 uild intelligent machines Understand Learning Learning is the gateway to understanding intelligence

13 (Artifical) Intelligence: A Working Definition Turing test Ingredients for AI! natural language processing! knowledge representation! automated reasoning! machine learning! computer vision! robotics to manipulate

14 Exciting Days for AI Artificial intelligence systems have recently seen some striking successes Learning

15 ML and AI Machine Learning systems are trained on examples rather than being programmed

16 Data Unsupervised Feature Learning lassic Supervised Learning Learning Structure.. Dynamic Learning Machine Learning Problems and Approaches!

17 asic Setting: lassification (x,y ),...,(x n,y n ) x i 2 R p and y i 2 Y = {, }, i =,...,n X n = x x p..... x n x p n A Y n = y. y n A

18 Genomics n patients p gene expression measurements... ; X n = x x p..... x n x p n A; Y n = y. y n A...

19 Text lassification: ag of Words X n = x x p..... x n x p n A

20 Image lassification......

21 Image lassification X n = x x p..... x n x p n A

22 Video lassification: Action Recognition

23 From classification to regression (x,y ),...,(x n,y n ) x i 2 R D and y i 2 Y = {, }, i =,...,n y i 2 Y 2 R, i =,...,n housing prices Living area (feet 2 ) Price ($s) price (in $) Living area (feet 2 ) #bedrooms Price ($s) square feet y i = f(x i )+ " i, > e.g. f(x) =w T x, " i N(, )

24 ! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches

25 Variations on a Theme (x,y ),...,(x n,y n ) Multiclass: x i 2 R D and y i 2 Y = {,...,T}, i =,...,n Multitask: x i 2 R D and y i 2 R T, i =,...,n Learning a similarity function (x,x ; y, ), (x,x 2 ; y,2 )...,(x n,x n ; y n,n ) x j,x i 2 R D and y i,j 2 [, ], j, i =,...,n

26 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A

27 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A

28 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A

29 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A

30 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A

31 Semisupervised Learning

32 Semisupervised Learning n = x x p..... x n x p n A Y n = ; y. y n A [ X u = x x p..... x u x p u A Manifold Learning

33 Unsupervised Learning X n = x x p..... x n x p n A Y n = y. y n A Given x,...,x n Goal: Extract patterns... lustering, k-means! Vector Quantization! Dimensionality reduction!...

34 ! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...!! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches

35 ! Learning Problems Supervised Learning Unsupervised Reinforcement Learning...! Learning types atch Learning Online Active...! Machine Learning: Problems and Approaches

36 Online/Incremental Learning (x,y ),...,(x n,y n ) f f (x,y ) (x 2,y 2 ) f 2... (x n,y n ) f n (x,y ) f...

37 The final estimator is constructed assembling the estimate away from th boundary obtained in the preview step with the estimate in the vicinity of th f Active Learning Learner can query points (x,y ) f (x 2,y 2 ) f 2... (a) (b) (x n,y n ) f n... (c) (d) Figure 8.7. The two step procedure for d =2:(a)InitialunprunedRDPandn/2 sample (b) Preview step RDP. Note that the cell with the arrow was pruned, but it contains a part of th boundary. (c) Additional sampling for the refinement step. (d) Refinement step.

38 Summary ``Learning is the acquisition of knowledge or skills through study, experience, or being taught We look for systems that are trained, rather than programmed, to perform a task! Learning from examples is a unifying paradigm in learning:!