Machine Learning: Statistical Methods  I


 Russell Flowers
 2 years ago
 Views:
Transcription
1 Machine Learning: Statistical Methods  I Introduction Stefan Roth, Department of Computer Science GRIS
2 Machine Learning I Lecturer: Prof. Stefan Roth, Ph.D. <sroth AT cs.tu...> About me... Teaching Assistant: Qi Gao <qgao AT gris.tu...> Announcements: Course web page: Mailing list: see subscription information on web page. Forum: Stefan Roth, Department of Computer Science GRIS 2
3 Course language will be English. This applies to lectures, exercises, announcements, etc. Why? Essentially all machine learning publications and books are written in English. Knowing the original terms is crucial. If strongly preferred, you may contact the course staff in German. English is encouraged though, because we may use your (anonymized) question to clarify points to the entire class. Stefan Roth, Department of Computer Science GRIS 3
4 Organization Lecture: 2 hours a week Wednesdays, 9:5011:30, S3/05 room 073 We will cover the foundational aspects of each topic. Exercise: 2 hours a week Wednesdays, 11:4013:20, S3/05 room 073 We will cover some practical aspects, and discuss the homework assignments. Stefan Roth, Department of Computer Science GRIS 4
5 Exam & Bonus System Most likely there will be an oral exam. Likely during the first week after classes end. Can be taken in English or German. Details depend on how many students end up taking the class. There will be a bonus of up to a full grade for those who do well in the homework assignments. Details TBA. Exercises: In order to get credit for 2+2 SWS, you need to actively participate in the exercises / turn in the homework assignments. If you do not hand in homework assignments regularly, you can only get credit for the lecture part (2 SWS). Stefan Roth, Department of Computer Science GRIS 5
6 Style Lectures: I would like the lectures to be at least partly interactive. Maybe more interactive than you are used to. This is supposed to be helpful for you and me. You are encouraged to ask questions! Exercises: Mostly interactive. You are encouraged to ask detailed questions! Your participation counts: Bonus for final exam. Stefan Roth, Department of Computer Science GRIS 6
7 Homework assignments Mix of written and programming assignments. We will have around 45 assignments. Programming assignments in MATLAB, standard environment for scientific computing.  Goal: Work with some real data to get a first hand knowledge of how the techniques work that we will learn.  Introduction during first exercise (next week). Also pen and paper exercises. The last assignment may be a larger projectlike one: Stay tuned... Stefan Roth, Department of Computer Science GRIS 7
8 Readings Course book: Christopher Bishop: Pattern Recognition and Machine Learning. Springer, ISBN (very good book, but not an easy read). Should be on reserve in the library. Other useful books: Duda, Hart & Stork: Pattern Classification, WileyInterscience, 2000, 2nd edition. ISBN (new version of a classic). David J. C. MacKay: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, ISBN (free download at Gelman et al.: Bayesian Data Analysis. CRC Press, 2nd ed., 2004, ISBN X Hastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, ISBN (the statistical perspective). Tom Mitchell: Machine Learning, McGrawHill, 1997, ISBN (classic, but getting outdated). Stefan Roth, Department of Computer Science GRIS 8
9 Readings Additional readings: At times I will post papers and tutorials. Will be available or linked from the course web page. I will often assign weekly readings: Please read them and come to class prepared! The Bishop book is a good investment, because it is also a very useful reference. Stefan Roth, Department of Computer Science GRIS 9
10 How does it fit into your course plan? Diplom: Anwendungsbezogene Informatik Possibly Praktische or Theoretische Informatik if you can find someone who will count ML I toward this.  Note that I will not be able to offer an exam in Theoretische Informatik. B.Sc. / M.Sc.: Human Computer Systems (see Modulhandbuch) Not Data Knowledge Engineering If you are strongly interested in machine learning you should:  Take ML: Statistical Methods for HCS credit and  Take ML: Symbolische Methoden for DKE credit Stefan Roth, Department of Computer Science GRIS 10
11 How does it fit into your course plan? Related classes: Human Computer Systems (WS, Schiele/Fellner): prerequisite Machine Learning: Statistical Methods II (WS, Schiele) Computer Vision I (SS, Schiele/Schindler)  II (WS, Roth) Maschinelles Lernen  Symbolische Verfahren (WS, Fürnkranz) Bildverarbeitung (SS, Sakas) Theses and projects: Topics in machine learning with applications in computer vision. Please talk to me if you are interested. Stefan Roth, Department of Computer Science GRIS 11
12 Machine Learning What is ML? What is its goal? Develop a machine / an algorithm that learns to perform a task from past experience. Why? What for? Fundamental component of every intelligent and/or autonomous system Discovering rules and patterns in data Automatic adaptation of systems Attempting to understand human / biological learning Stefan Roth, Department of Computer Science GRIS 12
13 Machine Learning in Action Stefan Roth, Department of Computer Science GRIS 13
14 Machine Learning: Examples Example 1: Recognition of handwritten digits These digits are given to us as small digital images. We have to build a machine to decide which digit it is. Obvious challenge: There are many different ways in which people handwrite. Stefan Roth, Department of Computer Science GRIS 14
15 Machine Learning: Examples Example 2: Classification of fish salmon sea bass count FIGURE 1.1. The objects to be classified are first sensed by a transducer (camera), whose signals are preprocessed. Next the features are extracted and finally the classification is emitted, here either salmon or sea bass. Although the information flow is often chosen to be from the source to the classifier, some systems employ information flow in which earlier levels of processing can be altered based on the tentative or preliminary response in later levels (gray arrows). Yet others combine two or more stages into a unified step, such as simultaneous segmentation and feature extraction. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc.! Stefan Roth, Department of Computer Science GRIS l* length FIGURE 1.2. Histograms for the length feature for the two categories. No single threshold value of the length will serve to unambiguously discriminate between the two categories; using length alone, we will have some errors. The value marked l will lead to the smallest number of errors, on average. From: Richard O. Duda, Peter E. Hart, and c 2001 by John Wiley & Sons, Inc. David G. Stork, Pattern Classification. Copyright "
16 Machine Learning: Examples More examples: filtering: Speech recognition: Vehicle control: Stefan Roth, Department of Computer Science GRIS 16
17 Impact & Successes Recognition of speech, letters, faces,... Autonomous vehicle navigation Games Backgammon worldchampion Chess: DeepBlue vs. Kasparov Google Finding new astronomical structures Fraud detection (credit card applications)... Stefan Roth, Department of Computer Science GRIS 17
18 Machine Learning Develop a machine / an algorithm that learns to perform a task from past experience. Put more abstractly: Our task is to learn a mapping from input to output. Put differently, we want to predict the output from the input. f : I O y = f(x; θ) Input: Output: Parameter(s): x I y O θ Θ images, text, other measurements,... (that is/are being learned ) Stefan Roth, Department of Computer Science GRIS 18
19 Classification vs. Regression Classification: Learn a mapping into a discrete space, e.g.: O = {0, 1} O = {0, 1, 2, 3,...} O = {verb, noun, nounphrase,...} Examples: Spam / not spam, sea bass vs. salmon, parsing a sentence, recognizing digits, etc. Stefan Roth, Department of Computer Science GRIS 19
20 Classification vs. Regression Regression: Learn a mapping into a continuous space, e.g.: O = R O = R 3 Examples: Curve fitting, financial analysis, Stefan Roth, Department of Computer Science GRIS 20
21 General Paradigm Training: Training data learn model θ Learned parameters Testing: predict 0, 1, 2, 8, 4, 6, 6, 7, 8, 9 Test data different from training data Predicted output Stefan Roth, Department of Computer Science GRIS 21
22 What data do we have for training? Data with labels (input / output pairs): supervised learning E.g. image with digit label Sensory data for car with intended steering control Data without labels: unsupervised learning E.g. automatic clustering (grouping) of sounds Clustering of text according to topics Data with and without labels: semisupervised learning No examples: learningbydoing Reinforcement learning Stefan Roth, Department of Computer Science GRIS 22
23 Some Key Challenges We need generalization! We cannot simply memorize the training set. What if we see an input that we haven t seen before? Different shape of the digit image (unknown writer) Dirt on the picture, etc. We need to learn what is important for carrying out our task. This is one of the most crucial points that we will return to many times. Stefan Roth, Department of Computer Science GRIS 23
24 Generalization How do we achieve generalization? width salmon sea bass 19 18? lightness FIGURE 1.5. Overly complex models for the fish will lead to decision boundaries that are complicated. While such a decision may lead to perfect classification of our training samples, it would lead to poor performance on future patterns. The novel test point marked? is evidently most likely a salmon, whereas the complex decision boundary shown leads it to be classified as a sea bass. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc. Stefan Roth, Department of Computer Science GRIS 24
25 Generalization How do we achieve generalization? width salmon sea bass Occam s Razor lightness FIGURE 1.6. The decision boundary shown might represent the optimal tradeoff between performance on the training set and simplicity of classifier, thereby giving the highest accuracy on new patterns. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc. We should not make the model overly complex! Stefan Roth, Department of Computer Science GRIS 25
26 Some Key Challenges Input x: Features: Choosing the right features is very important. Coding and use of domain knowledge. May allow for invariance (e.g. volume and pitch of voice). Curse of Dimensionality: If the features are too highdimensional, we will run into trouble  more later. Dimensionality reduction Stefan Roth, Department of Computer Science GRIS 26
27 Some Key Challenges How do we measure performance? 99% correct classification in speech recognition: What does that really mean? We understand the meaning of the sentence? We understand every word? For all speakers? Need more concrete numbers: % of correctly classified letters average distance driven (until accident...) % of games won % correctly recognized words, sentences, etc. Training vs. testing performance! Stefan Roth, Department of Computer Science GRIS 27
28 Some Key Challenges We also need to define the right error metric: Whis is better? Euclidean distance (L2 norm) might be useless. Stefan Roth, Department of Computer Science GRIS 28
29 Some Key Challenges Which is the right model? The learned parameters can mean a lot of different things.  w: may characterize the family of functions or the model space  w: may index the hypothesis space  w: vector, adjacency matrix, graph,... Stefan Roth, Department of Computer Science GRIS 29
30 Some Key Challenges Even if we have solved the other problems, computation is usually quite hard: Learning often involves some kind of optimization Find (search) best model parameters Often we have to deal with thousands of training examples Given a model, compute the prediction efficiently Stefan Roth, Department of Computer Science GRIS 30
31 Why is machine learning interesting (for you)? Machine learning is a challenging problem that is far from being solved. Our learning systems are primitive compared to us humans. Think about what and how quickly a child can learn! It combines insights and tools from many fields and disciplines: Traditional artificial intelligence (logic, semantic networks,...) Statistics Complexity theory Artificial neural networks Psychology Adaptive control,... Stefan Roth, Department of Computer Science GRIS 31
32 Why is machine learning interesting (for you)? Allows you to apply theoretical skills that you may otherwise only use rarely. Has lots of applications: Computer vision Computer linguistics Search (think Google) Digital assistants Computer systems... It is a growing field: Many major companies are hiring people with machine learning knowledge. Anecdote: At a recent workshop on computer graphics, about 2/3 of the groups said they would benefit from more machine learning knowledge. Stefan Roth, Department of Computer Science GRIS 32
33 Preliminary Syllabus Subject to change! Fundamentals (~ 3 weeks) Bayes decision theory, maximum likelihood, Bayesian inference Performance evaluation Probability density estimation Mixture models, expectation maximization Linear Methods (~ 34 weeks) Linear regression PCA, robust PCA Fisher linear discriminant generalized linear models Stefan Roth, Department of Computer Science GRIS 33
34 Preliminary Syllabus LargeMargin Methods (~ 34 weeks) Statistical learning theory Support vector machines Kernel methods Miscellaneous (~ 3 weeks) Model averaging (bagging & boosting) Graphical models (basic introduction) Stefan Roth, Department of Computer Science GRIS 34
35 Credits Large parts of the lecture material have been developed by Prof. Bernt Schiele for the previous iterations of this course. Many figures that I will use are directly taken out of the books by Chris Bishop and Duda, Hart & Stork. Stefan Roth, Department of Computer Science GRIS 35
36 Brief Review of Basic Probability What you should already know: F = a F = o Random picking: Red box: 60% of the time B = r B = b Blue box: 40% of the time Pick fruit from a box with equal probability p(b = r) = 0.6 p(b = b) = 0.4 p(f = a B = r) =p(f = o B = b) = 0.25 p(f = o B = r) =p(f = a B = b) = 0.75 Stefan Roth, Department of Computer Science GRIS 36
37 Brief Review of Basic Probability We usually do not mention the random variable (RV) explicitly (for brevity). Instead of p(x = x) we write: p(x) if we want to denote the probability distribution for a particular random variable X. p(x) if we want to denote the value of the probability of the random variable being x. It should be obvious from the context when we mean the random variable itself and a value that the random variable can take. Some people use upper case P (X = x) probability distributions. I usually don t. for (discrete) Stefan Roth, Department of Computer Science GRIS 37
38 Brief Review of Basic Probability Joint probability: The probability distribution of random variables X and Y taking on a configuration jointly. For example: Conditional probability: p(x, Y ) p(b = b, F = o) p(x Y ) The probability distribution of random variable X that random variable Y takes on a specific value For example: p(b = b F = o) given the fact Stefan Roth, Department of Computer Science GRIS 38
39 Basic Rules I Probabilities are always nonnegative: p(x) 0 Probabilities sum to 1: p(x) = 1 0 p(x) 1 x Sum rule or marginalization: p(x) = y p(x, y) p(y) = x p(x, y) p(x) and p(y) are called marginal distributions of the joint distribution p(x, y) Stefan Roth, Department of Computer Science GRIS 39
40 Basic Rules II Product rule: p(x, y) =p(x y)p(y) =p(y x)p(x) From this we directly follow... Bayes rule or Bayes theorem: We will widely use these rules. p(y x) = p(x y)p(y) p(x) Rev. Thomas Bayes Stefan Roth, Department of Computer Science GRIS 40
41 Continuous RVs What if we have continuous random variables, X = x R say? Any single value has zero probability. We can only assign a probability for a random variable being in a range of values: Pr(x 0 < X < x 1 )=Pr(x 0 X x 1 ) Instead we use the probability density p(x) Pr(x 0 X x 1 )= x1 x 0 p(x) dx Cumulative distribution function: P (z) = z p(x) dx and P (x) =p(x) Stefan Roth, Department of Computer Science GRIS 41
42 Continuous RVs p(x) P (x) Probability density function = pdf δx x Cumulative distribution function = cdf We can work with a density (pdf) as if it was a probability distribution: For simplicity we usually use the same notation for both. Stefan Roth, Department of Computer Science GRIS 42
43 Basic rules for pdfs What are the rules? Nonnegativity: Summing to 1: p(x) 0 p(x) dx =1 But: Marginalization: p(x) = in general p(x, y) dy p(y) = p(x, y) dx p(x) 1 Product rule: p(x, y) =p(x y)p(y) =p(y x)p(x) Stefan Roth, Department of Computer Science GRIS 43
44 Expectations The average value of a function f(x) distribution p(x) is the expectation: under a probability E[f] =E[f(x)] = x f(x)p(x) or E[f] = f(x)p(x) dx For joint distributions we sometimes write: E x [f(x, y)] Conditional expectation: E x y [f] =E x [f y] = x f(x)p(x y) Stefan Roth, Department of Computer Science GRIS 44
45 Variance and Covariance Variance of a single RV: var[x] =E [ (x E[x]) 2] = E[x 2 ] E[x] 2 Covariance of two RVs: cov(x, y) =E x,y [(x E[x])(y E[y])] = E x,y [xy] E[x]E[y] Random vectors: All we have said so far not only applies to scalar random variables, but also to random vectors. In particular, we have the covariance matrix: cov(x, y) = E x,y [ (x E[x])(y E[y]) T ] = E x,y [xy T ] E[x]E[y] T Stefan Roth, Department of Computer Science GRIS 45
46 Preview Review of some basics about probability Bayesian decision theory Loss functions Disclaimer: It will get quite a bit more mathematical than this :) Don t get scared away, but be aware that this will not be a walk in the park. Stefan Roth, Department of Computer Science GRIS 46
47 Readings for next week Introduction to ML: Bishop 1.0, 1.1 Review of the basics of probability: Bishop 1.2 (you can skip and for now) Decision theory: Bishop 1.5 For the curious: Probability: You could also look at MacKay 2 Brush up on information theory: Bishop 1.6 Stefan Roth, Department of Computer Science GRIS 47
203.4770: Introduction to Machine Learning Dr. Rita Osadchy
203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:
More informationIntroduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu
Introduction to Machine Learning Lecture 1 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction Logistics Prerequisites: basics concepts needed in probability and statistics
More informationMachine Learning. 01  Introduction
Machine Learning 01  Introduction Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge
More informationIntroduction to Pattern Recognition
Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationLearning is a very general term denoting the way in which agents:
What is learning? Learning is a very general term denoting the way in which agents: Acquire and organize knowledge (by building, modifying and organizing internal representations of some external reality);
More informationNetwork Machine Learning Research Group. Intended status: Informational October 19, 2015 Expires: April 21, 2016
Network Machine Learning Research Group S. Jiang InternetDraft Huawei Technologies Co., Ltd Intended status: Informational October 19, 2015 Expires: April 21, 2016 Abstract Network Machine Learning draftjiangnmlrgnetworkmachinelearning00
More informationMachine Learning and Data Mining. Fundamentals, robotics, recognition
Machine Learning and Data Mining Fundamentals, robotics, recognition Machine Learning, Data Mining, Knowledge Discovery in Data Bases Their mutual relations Data Mining, Knowledge Discovery in Databases,
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationMACHINE LEARNING IN HIGH ENERGY PHYSICS
MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!
More informationMA2823: Foundations of Machine Learning
MA2823: Foundations of Machine Learning École Centrale Paris Fall 2015 ChloéAgathe Azencot Centre for Computational Biology, Mines ParisTech chloe agathe.azencott@mines paristech.fr TAs: Jiaqian Yu jiaqian.yu@centralesupelec.fr
More informationMachine Learning CS 6830. Lecture 01. Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu
Machine Learning CS 6830 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu What is Learning? MerriamWebster: learn = to acquire knowledge, understanding, or skill
More informationIntroduction to Machine Learning Using Python. Vikram Kamath
Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression
More informationCPSC 340: Machine Learning and Data Mining. Mark Schmidt University of British Columbia Fall 2015
CPSC 340: Machine Learning and Data Mining Mark Schmidt University of British Columbia Fall 2015 Outline 1) Intro to Machine Learning and Data Mining: Big data phenomenon and types of data. Definitions
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation:  Feature vector X,  qualitative response Y, taking values in C
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationMachine Learning. Chapter 18, 21. Some material adopted from notes by Chuck Dyer
Machine Learning Chapter 18, 21 Some material adopted from notes by Chuck Dyer What is learning? Learning denotes changes in a system that... enable a system to do the same task more efficiently the next
More informationGovernment of Russian Federation. Faculty of Computer Science School of Data Analysis and Artificial Intelligence
Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education National Research University «Higher School of Economics» Faculty of Computer Science School
More informationMachine Learning. CUNY Graduate Center, Spring 2013. Professor Liang Huang. huang@cs.qc.cuny.edu
Machine Learning CUNY Graduate Center, Spring 2013 Professor Liang Huang huang@cs.qc.cuny.edu http://acl.cs.qc.edu/~lhuang/teaching/machinelearning Logistics Lectures M 9:3011:30 am Room 4419 Personnel
More informationDefinition: Suppose that two random variables, either continuous or discrete, X and Y have joint density
HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,
More informationQuestion 2 Naïve Bayes (16 points)
Question 2 Naïve Bayes (16 points) About 2/3 of your email is spam so you downloaded an open source spam filter based on word occurrences that uses the Naive Bayes classifier. Assume you collected the
More informationData Mining  Evaluation of Classifiers
Data Mining  Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010
More informationMachine Learning and Pattern Recognition Logistic Regression
Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,
More informationAttribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)
Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision
More information6.2.8 Neural networks for data mining
6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural
More informationPattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
More informationMaschinelles Lernen mit MATLAB
Maschinelles Lernen mit MATLAB Jérémy Huard Applikationsingenieur The MathWorks GmbH 2015 The MathWorks, Inc. 1 Machine Learning is Everywhere Image Recognition Speech Recognition Stock Prediction Medical
More informationMachine Learning and Statistics: What s the Connection?
Machine Learning and Statistics: What s the Connection? Institute for Adaptive and Neural Computation School of Informatics, University of Edinburgh, UK August 2006 Outline The roots of machine learning
More informationKATE GLEASON COLLEGE OF ENGINEERING. John D. Hromi Center for Quality and Applied Statistics
ROCHESTER INSTITUTE OF TECHNOLOGY COURSE OUTLINE FORM KATE GLEASON COLLEGE OF ENGINEERING John D. Hromi Center for Quality and Applied Statistics NEW (or REVISED) COURSE (KGCOE CQAS 747 Principles of
More informationFaculty of Science School of Mathematics and Statistics
Faculty of Science School of Mathematics and Statistics MATH5836 Data Mining and its Business Applications Semester 1, 2014 CRICOS Provider No: 00098G MATH5836 Course Outline Information about the course
More informationKickoff: Anomaly Detection Challenges
Kickoff: Anomaly Detection Challenges A Practical Course in SS2014 Huang Xiao Han Xiao Chair of IT Security (I20) Department of Informatics Technische Universität München January 31, 2014 Huang Xiao, Han
More informationClassification Techniques for Remote Sensing
Classification Techniques for Remote Sensing Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara saksoy@cs.bilkent.edu.tr http://www.cs.bilkent.edu.tr/ saksoy/courses/cs551
More informationModelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationMachine Learning Capacity and Performance Analysis and R
Machine Learning and R May 3, 11 30 25 15 10 5 25 15 10 5 30 25 15 10 5 0 2 4 6 8 101214161822 0 2 4 6 8 101214161822 0 2 4 6 8 101214161822 100 80 60 40 100 80 60 40 100 80 60 40 30 25 15 10 5 25 15 10
More informationMachine Learning. Mausam (based on slides by Tom Mitchell, Oren Etzioni and Pedro Domingos)
Machine Learning Mausam (based on slides by Tom Mitchell, Oren Etzioni and Pedro Domingos) What Is Machine Learning? A computer program is said to learn from experience E with respect to some class of
More informationCSE 517A MACHINE LEARNING INTRODUCTION
CSE 517A MACHINE LEARNING INTRODUCTION Spring 2016 Marion Neumann Contents in these slides may be subject to copyright. Some materials are adopted from Killian Weinberger. Thanks, Killian! Machine Learning
More informationLecture Slides for INTRODUCTION TO. ETHEM ALPAYDIN The MIT Press, 2004. Lab Class and literature. Friday, 9.00 10.00, Harburger Schloßstr.
Lecture Slides for INTRODUCTION TO Machine Learning ETHEM ALPAYDIN The MIT Press, 2004 alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml Lab Class and literature Friday, 9.00 10.00, Harburger
More informationPart III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing
CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and
More informationT61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier. Santosh Tirunagari : 245577
T61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier Santosh Tirunagari : 245577 January 20, 2011 Abstract This term project gives a solution how to classify an email as spam or
More informationTIETS34 Seminar: Data Mining on Biometric identification
TIETS34 Seminar: Data Mining on Biometric identification Youming Zhang Computer Science, School of Information Sciences, 33014 University of Tampere, Finland Youming.Zhang@uta.fi Course Description Content
More informationCSCI567 Machine Learning (Fall 2014)
CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /
More informationMVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr
Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Department of Applied Mathematics Ecole Centrale Paris Galen
More information8. Machine Learning Applied Artificial Intelligence
8. Machine Learning Applied Artificial Intelligence Prof. Dr. Bernhard Humm Faculty of Computer Science Hochschule Darmstadt University of Applied Sciences 1 Retrospective Natural Language Processing Name
More informationMachine learning for algo trading
Machine learning for algo trading An introduction for nonmathematicians Dr. Aly Kassam Overview High level introduction to machine learning A machine learning bestiary What has all this got to do with
More informationSupervised Learning (Big Data Analytics)
Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used
More informationCHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES
CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationComponent Ordering in Independent Component Analysis Based on Data Power
Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals
More informationHT2015: SC4 Statistical Data Mining and Machine Learning
HT2015: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Bayesian Nonparametrics Parametric vs Nonparametric
More informationData Clustering. Dec 2nd, 2013 Kyrylo Bessonov
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms kmeans Hierarchical Main
More informationAnalysis Tools and Libraries for BigData
+ Analysis Tools and Libraries for BigData Lecture 02 Abhijit Bendale + Office Hours 2 n Terry Boult (Waiting to Confirm) n Abhijit Bendale (Tue 2:45 to 4:45 pm). Best if you email me in advance, but I
More informationAn Overview of Knowledge Discovery Database and Data mining Techniques
An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,
More information1. Let A, B and C are three events such that P(A) = 0.45, P(B) = 0.30, P(C) = 0.35,
1. Let A, B and C are three events such that PA =.4, PB =.3, PC =.3, P A B =.6, P A C =.6, P B C =., P A B C =.7. a Compute P A B, P A C, P B C. b Compute P A B C. c Compute the probability that exactly
More informationMachine Learning in Spam Filtering
Machine Learning in Spam Filtering A Crash Course in ML Konstantin Tretyakov kt@ut.ee Institute of Computer Science, University of Tartu Overview Spam is Evil ML for Spam Filtering: General Idea, Problems.
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationMACHINE LEARNING BASICS WITH R
MACHINE LEARNING [Handson Introduction of Supervised Machine Learning Methods] DURATION 2 DAY The field of machine learning is concerned with the question of how to construct computer programs that automatically
More informationMachine Learning with MATLAB David Willingham Application Engineer
Machine Learning with MATLAB David Willingham Application Engineer 2014 The MathWorks, Inc. 1 Goals Overview of machine learning Machine learning models & techniques available in MATLAB Streamlining the
More informationOutlier Ensembles. Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598. Keynote, Outlier Detection and Description Workshop, 2013
Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier
More informationUsing Data Mining for Mobile Communication Clustering and Characterization
Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer
More informationEM Clustering Approach for MultiDimensional Analysis of Big Data Set
EM Clustering Approach for MultiDimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering Martin
More informationClassification Problems
Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems
More informationLearning outcomes. Knowledge and understanding. Competence and skills
Syllabus Master s Programme in Statistics and Data Mining 120 ECTS Credits Aim The rapid growth of databases provides scientists and business people with vast new resources. This programme meets the challenges
More informationAn Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015
An Introduction to Data Mining for Wind Power Management Spring 2015 Big Data World Every minute: Google receives over 4 million search queries Facebook users share almost 2.5 million pieces of content
More informationINTRODUCTION TO MACHINE LEARNING 3RD EDITION
ETHEM ALPAYDIN The MIT Press, 2014 Lecture Slides for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml3e CHAPTER 1: INTRODUCTION Big Data 3 Widespread
More informationAn Introduction to Data Mining
An Introduction to Intel Beijing wei.heng@intel.com January 17, 2014 Outline 1 DW Overview What is Notable Application of Conference, Software and Applications Major Process in 2 Major Tasks in Detail
More informationComparison of Nonlinear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data
CMPE 59H Comparison of Nonlinear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Nonlinear
More informationNeural Networks and Back Propagation Algorithm
Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important
More informationMachine Learning: Overview
Machine Learning: Overview Why Learning? Learning is a core of property of being intelligent. Hence Machine learning is a core subarea of Artificial Intelligence. There is a need for programs to behave
More informationInformation Management course
Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)
More informationPractical Data Science with Azure Machine Learning, SQL Data Mining, and R
Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationLecture 6. Artificial Neural Networks
Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm
More information01219211 Software Development Training Camp 1 (03) Prerequisite : 01204214 Program development skill enhancement camp, at least 48 personhours.
(International Program) 01219141 ObjectOriented Modeling and Programming 3 (30) Object concepts, objectoriented design and analysis, objectoriented analysis relating to developing conceptual models
More informationMachine Learning for Data Science (CS4786) Lecture 1
Machine Learning for Data Science (CS4786) Lecture 1 TuTh 10:10 to 11:25 AM Hollister B14 Instructors : Lillian Lee and Karthik Sridharan ROUGH DETAILS ABOUT THE COURSE Diagnostic assignment 0 is out:
More informationNew Ensemble Combination Scheme
New Ensemble Combination Scheme Namhyoung Kim, Youngdoo Son, and Jaewook Lee, Member, IEEE Abstract Recently many statistical learning techniques are successfully developed and used in several areas However,
More informationSTT315 Chapter 4 Random Variables & Probability Distributions KM. Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables
Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables Discrete vs. continuous random variables Examples of continuous distributions o Uniform o Exponential o Normal Recall: A random
More informationExploratory Data Analysis with MATLAB
Computer Science and Data Analysis Series Exploratory Data Analysis with MATLAB Second Edition Wendy L Martinez Angel R. Martinez Jeffrey L. Solka ( r ec) CRC Press VV J Taylor & Francis Group Boca Raton
More informationEmotion Detection from Speech
Emotion Detection from Speech 1. Introduction Although emotion detection from speech is a relatively new field of research, it has many potential applications. In humancomputer or humanhuman interaction
More informationChapter 6. The stacking ensemble approach
82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described
More informationNeural Networks Lesson 5  Cluster Analysis
Neural Networks Lesson 5  Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt.  Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29
More informationSTATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and
Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Webbased Analytics Table
More informationAn Introduction to Machine Learning
An Introduction to Machine Learning L5: Novelty Detection and Regression Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune,
More informationAnalytics on Big Data
Analytics on Big Data Riccardo Torlone Università Roma Tre Credits: Mohamed Eltabakh (WPI) Analytics The discovery and communication of meaningful patterns in data (Wikipedia) It relies on data analysis
More informationThe Chinese Restaurant Process
COS 597C: Bayesian nonparametrics Lecturer: David Blei Lecture # 1 Scribes: Peter Frazier, Indraneel Mukherjee September 21, 2007 In this first lecture, we begin by introducing the Chinese Restaurant Process.
More informationSupport Vector Machine. Tutorial. (and Statistical Learning Theory)
Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. jasonw@neclabs.com 1 Support Vector Machines: history SVMs introduced
More informationStatistical Validation and Data Analytics in ediscovery. Jesse Kornblum
Statistical Validation and Data Analytics in ediscovery Jesse Kornblum Administrivia Silence your mobile Interactive talk Please ask questions 2 Outline Introduction Big Questions What Makes Things Similar?
More informationFall 2012 Q530. Programming for Cognitive Science
Fall 2012 Q530 Programming for Cognitive Science Aimed at little or no programming experience. Improve your confidence and skills at: Writing code. Reading code. Understand the abilities and limitations
More informationJava Modules for Time Series Analysis
Java Modules for Time Series Analysis Agenda Clustering Nonnormal distributions Multifactor modeling Implied ratings Time series prediction 1. Clustering + Cluster 1 Synthetic Clustering + Time series
More informationMachine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
More informationSupport Vector Machines with Clustering for Training with Very Large Datasets
Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano
More informationLogistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression
Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max
More informationStatistics W4240: Data Mining Columbia University Spring, 2014
Statistics W4240: Data Mining Columbia University Spring, 2014 Version: January 30, 2014. The syllabus is subject to change, so look for the version with the most recent date. Course Description Massive
More informationClassification algorithm in Data mining: An Overview
Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department
More informationCOLUMBIA UNIVERSITY IN THE CITY OF NEW YORK DEPARTMENT OF INDUSTRIAL ENGINEERING AND OPERATIONS RESEARCH
Course: IEOR 4575 Business Analytics for Operations Research Lectures MW 2:403:55PM Instructor Prof. Guillermo Gallego Office Hours Tuesdays: 34pm Office: CEPSR 822 (8 th floor) Textbooks and Learning
More informationMachine Learning Logistic Regression
Machine Learning Logistic Regression Jeff Howbert Introduction to Machine Learning Winter 2012 1 Logistic regression Name is somewhat misleading. Really a technique for classification, not regression.
More information1 Maximum likelihood estimation
COS 424: Interacting with Data Lecturer: David Blei Lecture #4 Scribes: Wei Ho, Michael Ye February 14, 2008 1 Maximum likelihood estimation 1.1 MLE of a Bernoulli random variable (coin flips) Given N
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher
More informationMachine Learning Big Data using Map Reduce
Machine Learning Big Data using Map Reduce By Michael Bowles, PhD Where Does Big Data Come From? Web data (web logs, click histories) ecommerce applications (purchase histories) Retail purchase histories
More informationCSci 538 Articial Intelligence (Machine Learning and Data Analysis)
CSci 538 Articial Intelligence (Machine Learning and Data Analysis) Course Syllabus Fall 2015 Instructor Derek Harter, Ph.D., Associate Professor Department of Computer Science Texas A&M University  Commerce
More informationMS1b Statistical Data Mining
MS1b Statistical Data Mining Yee Whye Teh Department of Statistics Oxford http://www.stats.ox.ac.uk/~teh/datamining.html Outline Administrivia and Introduction Course Structure Syllabus Introduction to
More information