Administrivia. Traditional Recognition Approach. Overview. CMPSCI 370: Intro. to Computer Vision Deep learning

Transcription

1 : Intro. to Computer Vision Deep learning University of Massachusetts, Amherst April 19/21, 2016 Instructor: Subhransu Maji Finals (everyone) Thursday, May 5, 1-3pm, Hasbrouck 113 Final exam Tuesday, May 3, 4-5pm, Location: TBD (Review?) Syllabus includes everything taught after and including SIFT features. Lectures March 03 onwards. Honors section Administrivia Tuesday, April 26, 4-5pm 20 min presentation Friday, May 6, midnight writeup of 4-6 pages 2 Shallow vs. deep architectures Background Traditional neural networks Inspiration from neuroscience Stages of CNN architecture Visualizing CNNs State-of-the-art results Packages Overview Traditional Recognition Approach Image/ Video Pixels Hand-designed feature extraction Trainable classifier Object Class Features are not learned Trainable classifier is often generic (e.g. SVM) Many slides are by Rob Fergus and S. Lazebnik 3 4

2 Traditional Recognition Approach Features are key to recent progress in recognition Multitude of hand-designed features currently in use SIFT, HOG,. Where next? Better classifiers? Or keep building more features? What about learning the features? Learn a feature hierarchy all the way from pixels to classifier Each layer extracts features from the output of previous layer Train all layers jointly Image/ Video Pixels Layer 1 Layer 2 Layer 3 Simple Classifier Felzenszwalb, Girshick, McAllester and Ramanan, PAMI 2007 Yan & Huang (Winner of PASCAL 2010 classification competition) 5 6 Shallow vs. deep architectures Artificial neural networks Traditional recognition: Shallow architecture Image/ Video Pixels Hand-designed feature extraction Trainable classifier Object Class Image/ Video Pixels Deep learning: Deep architecture Layer 1 Layer N Simple classifier Object Class image credit wikipedia Artificial neural network is a group of interconnected nodes Circles here represent artificial neurons Note the directed arrows (denoting the flow of information) 7 8

3 Inspiration: Neuron cells Hubel/Wiesel Architecture D. Hubel and T. Wiesel (1959, 1962, Nobel Prize 1981) Visual cortex consists of a hierarchy of simple, complex, and hyper-complex cells 9 Source 10 Perceptron: a single neuron Example: Spam Basic unit of computation Input are feature values Each feature has a weight Sum in the activation Imagine 3 features (spam is positive class): free (number of occurrences of free ) money (number of occurrences of money ) BIAS (intercept, always has value 1) activation(w, x) = X i w i x i = w T x x w w T x If the activation is: > b, output class 1 otherwise, output class 2 x! (x, 1) w T x + b! (w,b) T (x, 1) w 1 x 1 w 2 x 2 w 3 x 3 CMPSCI /19 > b w T x > 0! SPAM!! CMPSCI /19

4 Geometry of the perceptron Two-layer network architecture In the space of feature vectors examples are points (in D dimensions) an weight vector is a hyperplane (a D-1 dimensional object) One side corresponds to y=+1 Other side corresponds to y=-1 y = vt h Non-linearity is important link function Perceptrons are also called as linear classifiers hi = f (wit x) w wt x = 0 CMPSCI 689 tanh(x) = 13 /19 The XOR function 1 e 1+e 2x 2x 14 Training ANNs Can a single neuron learn the XOR function? Exercise: come up with the parameters of a two layer network with two hidden units that computes the XOR function Here is a table for the XOR function Chain rule of gradient we know the desired output df(g(x))/dx = (df/dg)(dg/dx) Back-propagate the gradients to match the outputs Were too impractical till computers became faster

5 In the 1990s and early 2000s, simpler and faster learning methods such as linear classifiers, nearest neighbor classifiers, and decision trees were favored over ANNs. Why? Issues with ANNs Need many layers to learn good features many parameters need to be learned Needs vast amounts of training data (related to the earlier point) Training using gradient descent is slow, get stuck in local minima ANNs for vision The neocognitron, by Fukushima (1980) (But he didn t propose a way to learn these models) Convolutional Neural Networks Neural network with specialized connectivity structure Stack multiple stages of feature extractors Higher stages compute more global, more invariant features Classification layer at the end Convolutional Neural Networks Feed-forward feature extraction: 1. Convolve input with learned filters 2. Non-linearity 3. Spatial pooling 4. Normalization Supervised training of convolutional filters by back-propagating classification error Feature maps Normalization Spatial pooling Non-linearity Convolution (Learned) Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86(11): , Input Image 20

6 1. Convolution Dependencies are local Translation invariance Few parameters (filter weights) Stride can be greater than 1 (faster, less memory)... Per-element (independent) Options: Tanh Sigmoid: 1/(1+exp(-x)) Rectified linear unit (ReLU) - Simplifies backpropagation - Makes learning faster - Avoids saturation issues à Preferred option 2. Non-Linearity Input Feature Map Spatial Pooling Sum or max Non-overlapping / overlapping regions Role of pooling: Invariance to small transformations Larger receptive fields (see more of input) 4. Normalization Within or across feature maps Before or after spatial pooling Max Feature Maps Feature Maps After Contrast Normalization Sum 23 24

7 Compare: SIFT Descriptor Image Pixels CNN successes Lowe [IJCV 2004] Apply oriented filters Handwritten text/digits MNIST (0.17% error [Ciresan et al. 2011]) Arabic & Chinese [Ciresan et al. 2012] Simpler recognition benchmarks CIFAR-10 (9.3% error [Wan et al. 2013]) Traffic sign recognition Spatial pool (Sum) - Feature Vector Normalize to unit length 0.56% error vs 1.16% for humans [Ciresan et al. 2011] But until recently, less good at more complex datasets Caltech-101/256 (few training examples) ImageNet Challenge 2012 ImageNet Challenge 2012 [Deng et al. CVPR 2009] Similar framework to LeCun 98 but: Bigger model (7 hidden layers, 650,000 units, 60,000,000 params) More data (106 vs. 103 images) GPU implementation (50x speedup over CPU) Trained on two GPUs for a week Better regularization for training (DropOut) 14+ million labeled images, 20k classes Images gathered from Internet Human labels via Amazon Turk The challenge: 1.2 million training images, 1000 classes A. Krizhevsky, I. Sutskever, and G. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012 A. Krizhevsky, I. Sutskever, and G. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NIPS

8 ImageNet Challenge 2012 Visualizing CNNs Krizhevsky et al % error (top-5) Next best (SIFT + Fisher vectors) 26.2% error 30 Top-5 error rate % SuperVision ISI Oxford INRIA Amsterdam M. Zeiler and R. Fergus, Visualizing and Understanding Convolutional Networks, arxiv preprint, Layer 1 Filters Layer 1: Top-9 Patches Patches from validation images that give maximal activation of a given feature map Similar to the filter banks used for texture recognition 31 32

9 Layer 2: Top-9 Patches Layer 3: Top-9 Patches 33 Layer 4: Top-9 Patches 34 Layer 5: Top-9 Patches 35 36

10 Evolution of Features During Training Evolution of Features During Training Occlusion Experiment Mask parts of input with occluding square Monitor output (class probability) Total activation in most active 5 th layer feature map Other activations from same feature map 39 40

11 p(true class) Most probable class Total activation in most active 5 th layer feature map Other activations from same feature map p(true class) Most probable class Total activation in most active 5 th layer feature map Other activations from same feature map 43 44

12 ImageNet Classification 2013 Results Test error (top-5) CNNs for small datasets Take model trained on ImageNet Take outputs of 6 th or 7 th layer before or after nonlinearity as features Train linear SVMs on these features (like retraining the last layer of the network) Optionally back-propagate: fine-tune features and/or classifier on new dataset 0.1 Clarifai (extra data) NUS Andrew Howard UvA-Euvision Adobe CognitiveVision ImageNet Test error at 0.07 (Google & Oxford groups) Tapping off features at each Layer Plug features from each layer into linear SVM Results on benchmarks [1] Caltech-101 (30 samples per class) [1] SUN 397 dataset (DeCAF) [1] Caltech-UCSD Birds (DeCAF) [2] MIT-67 Indoor Scenes dataset (OverFeat) Higher layers are better 47 [1] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition, arxiv preprint, 2014 [2] A. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson, CNN Features off-the-shelf: an Astounding Baseline for Recognition, arxiv preprint,

13 CNN features for detection CNN features for face verification R-CNN achieves map of 53.7% on PASCAL VOC 2010 For comparison, Uijlings et al. (2013) report 35.1% map using the same region proposals, but with a spatial pyramid and bag-of-visual-words approach. Part-based model with HOG (DPM, Poselets) ~ 33.5% R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, CVPR Open-source CNN software Cuda-convnet (Alex Krizhevsky, Google) High speed convolutions on the GPU Caffe (Y. Jia, Berkeley) Replacement of deprecated Decaf High performance CNNs Flexible CPU/GPU computations Overfeat (NYU) MatConvNet (Andrea Vedaldi, Oxford) An easy to use toolbox for CNNs from MATLAB Comparable performance/features with Caffe 51 Y. Taigman, M. Yang, M. Ranzato, L. Wolf, DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR 2014, to appear. 50