Recent Trends in AI. Carsten Rother

Transcription

1 Recent Trends in AI Carsten Rother 03/02/2016

2 Prüfungsfragen Nur vom zweiten Teil der Vorlesung (Dimitri Schlesinger, Carsten Rother) Drei Typen von Aufgaben: 1) Algorithmen 2) Definitionen und Wissensfragen 3) Theoretische Herleitungen Fragen werden auf Deutsch gestellt 2

3 1) Algorithmen Was würde ein parallelisierter ICM Algorithmus in den nächsten zwei Schritten machen? Bitte zeichnen sie ein. Gegeben die Energy: θ 1 x 1 x 2 x 4 x 3 θ 4 θ 3 x i {0,1} θ 1 0 = 0, θ 1 1 = 1 θ 2 0 = 1, θ 2 1 = 1 θ 3 0 = 2, θ 3 1 = 1 θ 4 0 = 1, θ 4 1 = 2 θ ij x i, x j = x i x j For all i, j θ 23 θ 2 Initializer Zustand: x 1 =0 x 2 =1 x 4 =0 Schritt 1: x 3 = 1 x 1 =? x 2 =? x 4 =? x 3 =? Hinweis: dunkle Konten werden im ersten Schritt nicht verändert Schritt 2: x 1 =? x 2 =? x 4 =? x 3 =? 3

4 2) Definitionen und Wissensfragen Frage: Was bedeutet die Vorhersage einer strukturierten Ausgabe ( Structured Output Prediction ) Antwort: Die Ausgaben besteht aus Teilen die von einander abhängen. Der Zusammenhang der Teile wird modelliert. 4

5 3) Theoretische Herleitungen Man berechne die Wahrscheinlichkeit dafür, dass die Summe der Augenzahlen zweier voneinander unabhängig gewürfelten Spielwürfel durch 5 teilbar ist. (ähnliche Aufgaben wurden in der Vorlesung und Übung betrachtet) 5

6 Roadmap for this lectures The Boom of Artificial Intelligence (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 6

7 Roadmap for this lectures The Boom of Artificial Intelligence (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 7

8 Current Boom of AI Hal Varian, Chief Engineer of Google (2009): ''I keep saying the sexy job in the next ten years will be statisticians and machine learners. People think I am joking, but who would have guessed that computer engineers would have been the sexy job of the 1990s? The ability to take data, to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it, that is going to be a hugely important skill in the next decades.'' 8

9 Deep Learning Deep learning attempt to model high-level abstractions in data by using multiple processing layers with complex structures Very popular are feed-forward neural networks with many layers, e.g. up to 150 layers and 40+ million neurons 9

10 Deep learning and AI Deep learning techniques outperform humans for challenging cognitive tasks such as: What's shown in the image? ImageNet Challenge Many big companies open or run AI labs (Google, Facebook, Microsoft, etc). Combine different cognitive tasks, e.g. speech and image recognition, to built the human brain in a computer Open AI Lab (non-profit artificial intelligence research company) 10

11 Deep Learning [Deep Learning, Y. LeCun, Y. Bengio, G. Hinton, Nature article 2015] 11

12 AI for Autonomous Driving 12

13 AI for Art The Shipwreck of the Minotaur by J.M.W. Turner, [A Neural Algorithm of Artistic Style, Gatys, Ecker, Bethge. ArXiv 2015] 13

14 AI for Art The Starry Night by Vincent van Gogh, [A Neural Algorithm of Artistic Style, Gatys, Ecker, Bethge. ArXiv 2015] 14

15 AI for Art Composition VII by Wassily Kandinsky, 1913 [A Neural Algorithm of Artistic Style, Gatys, Ecker, Bethge. ArXiv 2015] 15

16 Chip Development Specialized Nvidia Chips for Deep Learning "NVIDIA's GPU is central to advances in deep learning and supercomputing. We are leveraging these to create the brain of future autonomous vehicles that will be continuously alert, and eventually achieve superhuman levels of situational awareness. Autonomous cars will bring increased safety, new convenient mobility services and even beautiful urban designs -- providing a powerful force for a better future." [Nvidia, ] (See more at: 16

17 Roadmap for this lectures The Boom of Artificial Inelegance (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 17

18 Convolutional Neural Networks [slide from 2 lectures ago] 18

19 Three Classification tasks Neural Network (small) Input signal {0,1,2, } Convolutional Neural Network (CNN) {person, car, background, } Convolutional Neural Network - pixelwise 19

20 Standard Feed-forward Neuronal Network x 1 h 1 1 h x 2 x 3 h 2 1 h 2 2 h sigmoid y [0, 1] h 3 1 h 3 2 x 4 max 0, W 1 T x + b 1 W 1 (4x3) matrix max 0, W 2 T h + b 2 W 2 (3x3) matrix max 0, W 3 T h + b 3 W 3 (3x1) matrix Trained with Backpropagation (differentiate error function wrt Parameters (W) 20

21 CNNs for Images person 21

22 How to deal with large Input Spaces? Images can have millions of pixels, i.e., x is very high dimensional Prohibitive to have fully-connected layer Idea: Statistics are similar at different locations (Lecun 1998) We can use a locally connected layer: Connect each hidden unit to a small input patch and share the weight across space This is called a convolution layer and the network is a convolutional network 22

23 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 23

24 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 24

25 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 25

26 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 26

27 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 27

28 Convolutional Layer Same Convolution Filter h j n = max(0, K k=1 h k n 1 w jk n ) 28

29 Convolutional Neural Networks (CNN) [Slide Credit: Sanja Fidler, Pic adopted from: A. Krizhevsky] 29

30 Convolutional Neural Networks (CNN) 30

31 Convolutional Neural Networks (CNN) 31

32 Convolutional Neural Networks (CNN) Let s make it deep: repeat many layers 32

33 Convolutional Neural Networks (CNN) 33

34 Convolutional Neural Networks (CNN) Represents a deep Filterbank Represents a classifier 34

35 Convolutional Neural Networks (CNN) Why does it work so well? 1) Use millions of data use tens of millions of data Wait for a week to finish training 2) Drop out training 3) Big GPU Cluster 35

36 Classification Performance ImageNet - main challenge for object classification: classes, 1.2M training images, 150K for test 36

37 Architecture for Classification 37

38 ImageNet results Top-5 error on ImageNet (1000 classes) 3 weeks ago 152 Layer architecture! 38

39 CNN Visualization 39

40 CNN Visualization Why are they so successful? Learning hierarchical features gives the boost 40

41 Pixel-level Output Semantic Segmentation 2013: ~45% correct (chance: 4.7%) 2015: ~77.8% correct [Result from Oxford University, Torr group] 41

42 Pixel-level Output: State-of-the-Art 42

43 CNNs for Dense Labelling [U-Net: Convolutional Networks for Biomedical Image Segmentation, Ronneberg et al MICCAI 2015] 43

44 Roadmap for this lectures The Boom of Artificial Inelegance (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 44

45 The important factors Model Power Performance (e.g. number layers in the neural network) Training Data (unlabeled and labeled!) More Training Data = More powerful models = Better Performance 45

46 Where does the training data come from? 46

47 Crowd Sourcing Task: Get segmentation of each cell for each frame 47

48 Crowd Sourcing Input (part of flywing) Ideal Output Images with User Annotation Costs: 300 Euro money for 600 small image requests to crowd 48

49 Search Engines Input: Tiger 49

50 Simulation Test Time: Depth Image Body Labelling Tracking in 3D Training Time: [Vicon] Record mocap 100,000s of poses Retarget a graphics body shapes Render (depth, body parts) pairs + add noise 50

51 3D Class Generation Class motorbikes 3D Shape Manifolds Class bycicles [Kyle, Mitra, Rother, Shotton, Torr, GCPR 2015] 51

52 Other tricks Data Augmentation to get more data Input label rotations Thin-plate spline deformation Etc. Network Re-used: Use Network from another task Retrain with small amount of traimning data 52

53 Roadmap for this lectures The Boom of Artificial Inelegance (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 53

54 Bio-Imaging Undirected Graphical Models x j θ ij (x i, x j ) x i θ i (x i ) C.elegans E x = i θ i x i + i,j N 4 Θ ij (x i, x j ) [D. Kainmüller, F. Jug, C. Rother, and G. Myers, Active Graph Matching for Automatic Joint Segmentation and Annotation of C. elegans, MICCAI 2014] 54

55 Task Input: 3D Data 55

56 Task Output: segmentation and naming of all nuclei 56

57 Training Data nuclei annotated (558 nuclei in total) in 30 worms 57

58 Training Data Motion Model from the Training Data Side View Top View 58

59 Test Data Preparation Input: 3D data Candidate Segmentation using Generalized Hough Transform with an ellipsoid as a template Test worm with e.g. 700 nuclei hypothesis 59

60 Matching Task Mean Training Data (Atlas) We will have a complex graphical model (only sketched here) a ij a ij Test Data a ij 0,1. a ij = 1 means that nuclei i from atlas is connected with nuclei j from test data Labelling a 0,1 A represents a full matching (A has maximum value 357*700) From a we can read out the name for each test data nuclei There are two constraints which have to be satisfied: 1. Each training nuclei can only be connected to a maximum of one test nuclei 2. Each test nuclei can only be connected to a maximum of one training nuclei 60

61 Matching Energy Averaged Position Mean Training Data (Atlas) a 20,j a 110,j a 290,j Test Data E a = i,j Nu θ ij a ij + i,j,k,l Np Θ ijkl a ij a kl + λ i a i Test Nuclei i Training Nuclei j How similar are they, θ ij [0, ] Confidence score for Test Nuclei i Minimize energy with a dual decomposition technique A large negative constant (why?) 61 a ij i j k a kl l Θ ijkl [0, ] Measures the similarity of the two red vectors

62 Matching Energy Neighborhood strcture N p Mean Training Data (Atlas) a 20,j a 110,j a 290,j Test Data E x = i,j Nu θ ij a ij + i,j,k,l Np Θ ijkl a ij a kl + λ i a i Test Nuclei i Training Nuclei j How similar are they, θ ij [0, ] Confidence score for Test Nuclei i Minimize energy with a dual decomposition technique A large negative constant (why?) 62 a ij i j k a kl l Θ ijkl [0, ] Measures the similarity of the two red vectors

63 Extension and Results Results: 86% accuracy [Ours] 62% accuracy without pairwise terms [Hungarian Matching] 86% with a lot of user effort [Long et al.] Mean Training Data (Atlas) White: correct match; Red: wrong match Test Data An extension where we guessed missing nuclei in atlas. Black: match to guessed nuclei. The result improves since fewer red links. 63

64 Extension and Results Extension: Also predict the global deformations of the test data with respect to atlas. This means to formulate an energy E(a, b, t) 64

65 Roadmap for this lectures The Boom of Artificial Inelegance (AI) Convolutional Neural Networks for Image Classification Where does the Training Data come from? Graphical Models for Bio-Imaging Lectures in Machine Learning and Computer Vision 65

66 Lectures WS 15/16 Computer Vision 1 (2+2) Machine Learning (2+2) SS 16 Computer Vision 2 (2+2) Optimization for Machine Learning (2+2) Image Processing (1+1) For doing a Master/PhD in the CVLD one should do the computer vision or machine learning track 66