Introduction to Computer Vision

Transcription

1 Reaching human-level AI? Introduction to Computer Vision Chen Yu Indiana University Terminator 2 (1991) Carolco Pictures Inc. What are expected from human users? Being adaptive, social, and considerate, for instance, a car that knows when you are sleepy and should take a break. glasses that can recognize the person you are talking to and whisper his/her name in your ear. Learning --- not directly programmed to solve a problem but being able to develop their own program based on examples. Why machine intelligence is a hard problem? Brainy tasks are easy while easy tasks are hard. If you imagine teaching a person in a closed, dark, soundproof box with only a telegraph-like connection (programming language), you can quickly realize how difficult it is for computers to become more intelligent and helpful. sensory input learning algorithms 1

2 Pinhole cameras First Known Photograph (1826) Pinhole camera box with a small hole in it Image is upside down, but not mirrored left to right Digital Image From Images to Understanding? The true fathers of digital photography, Willard S. Boyle and George E. Smith, invented the CCD. Charge Coupled Device (CCD): an array of individual light sensitive cells. One cell corresponds to one element in the whole picture, called a pixel

3 Color Space Face matching Pixelwise template matching? 3

4 Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes (Kevin Murphy, et. al 2003) 4

5 Why study Computer Vision? This means computer vision should involve inference. Just as human observers, machines must encode and use prior information about the world and the combination of evidence from multiple cues to infer what is in the world. Image and video are everywhere. Why study Computer Vision? Application of Computer Vision: First down line Engineering: building artificial intelligent systems. How do we build systems that perceive the visual world? How do we build systems that data mine video and image data? Application: Medical image, CSI, surveillance, entertainment Science: understanding natural intelligence. Various deep and attractive scientific mysteries How do we see? How does object recognition work? 5

6 Virtual Advertising (Augmented Reality) Application: plate reader at toll stations Tracking people More Tracking 6

7 Image Segmentation and Query Google Earth Image composition Face and license plate blurring technology Challenges There are lots of interesting questions, but few answers. The real world is 3D. The 2D intensity image is the result of a perspective projection of the 3D scene. When 3D objects are mapped into the camera plane by perspective projection, a lot of information disappears. edu/ The visual appearance of different instances of the same object type varies from one instance to the other. The visual appearance of the same instance of an object changes due to the factors such as illumination and occlusion. Objects People Recognition 7

8 Fingerprint Matching Machine Learning Herbert Simon: Learning is any process by which a system improves performance from experience. Many animals are capable of learning. Machine learning is programming computers to optimize a performance using example data or past experience (inference in past statistics). Why would we want computers to learn? Data Intensive Scientific Discovery: discover new knowledge from huge amounts of data (Jim Gray). It is too difficult or too expensive to program computers directly to perform a task. Solution changes in time (routing on a computer network). What is machine learning We write a parameterized program, and let the learning algorithm find the set of parameters (based on examples of inputs and outputs training data) that best approximates thedesired function or behavior. Solution needs to be adapted or customized (learn user s interests). 8

9 Three types of learning Supervised learning: given training examples of inputs and corresponding outputs (x1,y1), (x2,y2),, (xn,yn), produce the correct outputs for new inputs. Three types of learning Unsupervised learning: given only inputs, find structures and regularities in the data. Image segmentation 9

10 Three types of learning Reinforcement learning: an agent takes inputs from the environment, and takes actions that affect the environment. The agent gets a scalar reward or punishment. The goal of the agent is to learn to produce action sequences that maximize the expected reward. Generalization One way to build a learning system is to memorize all the training samples and corresponding outputs. Given a new input, we compare it with memorized samples and find the matching output. Problem: how to deal with unseen input? The ability to produce correct outputs on unseen inputs is called generalization. The big theoretical question in machine learning is how to get good generalization with a limited number of samples. One more step beyond memorization A simple way to make predictions is to just look at the training cases whose inputs are near the inputs for the test case. We might then assign the category of new input based on these nearby training cases to get our prediction for the target in the test case. 10

11 K nearest neighbors Step 1: determine k Step 2: calculate the distances between the new input and all the training data Step 3: sort the distance and determine k nearest neighbors based on the k th minimum distance. Step 4: gather the categories of those neighbors. Step 5: determine the category based on majority vote. Issues What is the right distance measurement? Euclidean How to combine neighbors labels? Majority vote Near neighbors should count more than far neighbors. Each neighbor casts vote with a weight, depending on the distance. How to choose K? odd numbers Parameter selection: How to determine K? The goal is to produce correct outputs on unseen inputs During training, given training set (x1,y1), (x2,y2),, (xn,yn). We write knn code. Now we have a classifier that can predict the output category based on a new input Xnew. Since we don t know what new input we will get, the only way to determine k is to use training data, more specially, measuring training set accuracy. 11

12 Parameter selection by training data During training, given training set (x1,y1), (x2,y2),, (xn,yn). We write knn code. Now we have a classifier that can predict the output category based on a new input Xnew. We try different values of K, 1, 3, 5, 9 We measure the accuracy on training examples in each case. We select K that maximizes the predicts on training data Overfitting It does an excellent job of fitting the training data points Overfitting is when a learning algorithm performs too good on the training set, compared to its true performance on unseen test data. Never use training accuracy to select parameters. It does not reflect the structure which we expect to be present in unseen data. Instead, overfitting also fits noise in training data, not the general underlying regularity. Overfitting The big theoretical question in machine learning is how to get good generalization with a limited number of samples. History of Machine Learning 60s: neural networks, pattern recognition, Kalman filter 70s: symbolic concept induction, etc 80s: decision tree, connectionisms, backpropagation, Hidden Markov Models, PCA, etc 90s: reinforcement learning, boosting, Bayes Net learning, etc 2000s: support vector machines, graphical models, co training, ISO MAP, relational learning, sampling techniques, etc 12

13 Computer Vision The time is ripe. Many basic effective algorithms available. Large amounts of data available. Large amounts of computing resources available. Findings on how human brain works. Machine Learning Go beyond computer vision Processing of text and speech data Data mining, pattern recognition, search engines, Textbook Two suggested: Sonka, M., Hlavac, V., & Boyle, R. (2007) Image Processing, Analysis, and Machine Vision. Brooks/Cole Publishing Company, ISBN X (2nd edition), ISBN X (3rd edition). Bishop, C. M. (2006) "Pattern Recognition and Machine Learning ", Springer. Method Mathematic method Calculus(derivatives,, etc.) Linear Algebra (matrix, eigenvalues,, etc.) Optimization methods (least squares,.., etc.) Probability theory Matlab C/C++ Programming method 13

14 Assignments Tuesdays Two types of assignments: Projects (two weeks): writing your own code and write research reports. Homework (one week): written assignment or the work based on using existing programs/functions, applying them on image data, and reporting the results. Final project Grading Class participation 10% Homework 20% 3 5 Projects 40% Final project 30% Collaboration Policy You may ask people for help with general concepts and basic programming but your work (including your code and report) must be your own. You areencouragedto encouraged acquirerelevantrelevant information from the web. The web is a great resource but don't be tempted to look for solutions to homework on the web. Be careful not to take code or text from the web. Anything you get from anywhere else should be scrupulously cited. Assignment 1 What computational issue (algorithms or applications, etc.) in computer vision is most challenging and interesting? Explain, in your own words, why the problem is hard and why the solution of the problem would lead to some breakthroughs, and whether, and if so how, human brain may be able to deal with it. Keep your answer to no more than 2 pages, including figures and text. 14