Machine Learning: Statistical Methods - I

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Machine Learning: Statistical Methods - I"

Transcription

1 Machine Learning: Statistical Methods - I Introduction Stefan Roth, Department of Computer Science GRIS

2 Machine Learning I Lecturer: Prof. Stefan Roth, Ph.D. <sroth AT cs.tu-...> About me... Teaching Assistant: Qi Gao <qgao AT gris.tu-...> Announcements: Course web page: Mailing list: see subscription information on web page. Forum: Stefan Roth, Department of Computer Science GRIS 2

3 Course language will be English. This applies to lectures, exercises, announcements, etc. Why? Essentially all machine learning publications and books are written in English. Knowing the original terms is crucial. If strongly preferred, you may contact the course staff in German. English is encouraged though, because we may use your (anonymized) question to clarify points to the entire class. Stefan Roth, Department of Computer Science GRIS 3

4 Organization Lecture: 2 hours a week Wednesdays, 9:50-11:30, S3/05 room 073 We will cover the foundational aspects of each topic. Exercise: 2 hours a week Wednesdays, 11:40-13:20, S3/05 room 073 We will cover some practical aspects, and discuss the homework assignments. Stefan Roth, Department of Computer Science GRIS 4

5 Exam & Bonus System Most likely there will be an oral exam. Likely during the first week after classes end. Can be taken in English or German. Details depend on how many students end up taking the class. There will be a bonus of up to a full grade for those who do well in the homework assignments. Details TBA. Exercises: In order to get credit for 2+2 SWS, you need to actively participate in the exercises / turn in the homework assignments. If you do not hand in homework assignments regularly, you can only get credit for the lecture part (2 SWS). Stefan Roth, Department of Computer Science GRIS 5

6 Style Lectures: I would like the lectures to be at least partly interactive. Maybe more interactive than you are used to. This is supposed to be helpful for you and me. You are encouraged to ask questions! Exercises: Mostly interactive. You are encouraged to ask detailed questions! Your participation counts: Bonus for final exam. Stefan Roth, Department of Computer Science GRIS 6

7 Homework assignments Mix of written and programming assignments. We will have around 4-5 assignments. Programming assignments in MATLAB, standard environment for scientific computing. - Goal: Work with some real data to get a first hand knowledge of how the techniques work that we will learn. - Introduction during first exercise (next week). Also pen and paper exercises. The last assignment may be a larger project-like one: Stay tuned... Stefan Roth, Department of Computer Science GRIS 7

8 Readings Course book: Christopher Bishop: Pattern Recognition and Machine Learning. Springer, ISBN (very good book, but not an easy read). Should be on reserve in the library. Other useful books: Duda, Hart & Stork: Pattern Classification, Wiley-Interscience, 2000, 2nd edition. ISBN (new version of a classic). David J. C. MacKay: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, ISBN (free download at Gelman et al.: Bayesian Data Analysis. CRC Press, 2nd ed., 2004, ISBN X Hastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, ISBN (the statistical perspective). Tom Mitchell: Machine Learning, McGraw-Hill, 1997, ISBN (classic, but getting outdated). Stefan Roth, Department of Computer Science GRIS 8

9 Readings Additional readings: At times I will post papers and tutorials. Will be available or linked from the course web page. I will often assign weekly readings: Please read them and come to class prepared! The Bishop book is a good investment, because it is also a very useful reference. Stefan Roth, Department of Computer Science GRIS 9

10 How does it fit into your course plan? Diplom: Anwendungsbezogene Informatik Possibly Praktische or Theoretische Informatik if you can find someone who will count ML I toward this. - Note that I will not be able to offer an exam in Theoretische Informatik. B.Sc. / M.Sc.: Human Computer Systems (see Modulhandbuch) Not Data Knowledge Engineering If you are strongly interested in machine learning you should: - Take ML: Statistical Methods for HCS credit and - Take ML: Symbolische Methoden for DKE credit Stefan Roth, Department of Computer Science GRIS 10

11 How does it fit into your course plan? Related classes: Human Computer Systems (WS, Schiele/Fellner): prerequisite Machine Learning: Statistical Methods II (WS, Schiele) Computer Vision I (SS, Schiele/Schindler) - II (WS, Roth) Maschinelles Lernen - Symbolische Verfahren (WS, Fürnkranz) Bildverarbeitung (SS, Sakas) Theses and projects: Topics in machine learning with applications in computer vision. Please talk to me if you are interested. Stefan Roth, Department of Computer Science GRIS 11

12 Machine Learning What is ML? What is its goal? Develop a machine / an algorithm that learns to perform a task from past experience. Why? What for? Fundamental component of every intelligent and/or autonomous system Discovering rules and patterns in data Automatic adaptation of systems Attempting to understand human / biological learning Stefan Roth, Department of Computer Science GRIS 12

13 Machine Learning in Action Stefan Roth, Department of Computer Science GRIS 13

14 Machine Learning: Examples Example 1: Recognition of handwritten digits These digits are given to us as small digital images. We have to build a machine to decide which digit it is. Obvious challenge: There are many different ways in which people handwrite. Stefan Roth, Department of Computer Science GRIS 14

15 Machine Learning: Examples Example 2: Classification of fish salmon sea bass count FIGURE 1.1. The objects to be classified are first sensed by a transducer (camera), whose signals are preprocessed. Next the features are extracted and finally the classification is emitted, here either salmon or sea bass. Although the information flow is often chosen to be from the source to the classifier, some systems employ information flow in which earlier levels of processing can be altered based on the tentative or preliminary response in later levels (gray arrows). Yet others combine two or more stages into a unified step, such as simultaneous segmentation and feature extraction. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc.! Stefan Roth, Department of Computer Science GRIS l* length FIGURE 1.2. Histograms for the length feature for the two categories. No single threshold value of the length will serve to unambiguously discriminate between the two categories; using length alone, we will have some errors. The value marked l will lead to the smallest number of errors, on average. From: Richard O. Duda, Peter E. Hart, and c 2001 by John Wiley & Sons, Inc. David G. Stork, Pattern Classification. Copyright "

16 Machine Learning: Examples More examples: filtering: Speech recognition: Vehicle control: Stefan Roth, Department of Computer Science GRIS 16

17 Impact & Successes Recognition of speech, letters, faces,... Autonomous vehicle navigation Games Backgammon world-champion Chess: Deep-Blue vs. Kasparov Google Finding new astronomical structures Fraud detection (credit card applications)... Stefan Roth, Department of Computer Science GRIS 17

18 Machine Learning Develop a machine / an algorithm that learns to perform a task from past experience. Put more abstractly: Our task is to learn a mapping from input to output. Put differently, we want to predict the output from the input. f : I O y = f(x; θ) Input: Output: Parameter(s): x I y O θ Θ images, text, other measurements,... (that is/are being learned ) Stefan Roth, Department of Computer Science GRIS 18

19 Classification vs. Regression Classification: Learn a mapping into a discrete space, e.g.: O = {0, 1} O = {0, 1, 2, 3,...} O = {verb, noun, nounphrase,...} Examples: Spam / not spam, sea bass vs. salmon, parsing a sentence, recognizing digits, etc. Stefan Roth, Department of Computer Science GRIS 19

20 Classification vs. Regression Regression: Learn a mapping into a continuous space, e.g.: O = R O = R 3 Examples: Curve fitting, financial analysis, Stefan Roth, Department of Computer Science GRIS 20

21 General Paradigm Training: Training data learn model θ Learned parameters Testing: predict 0, 1, 2, 8, 4, 6, 6, 7, 8, 9 Test data different from training data Predicted output Stefan Roth, Department of Computer Science GRIS 21

22 What data do we have for training? Data with labels (input / output pairs): supervised learning E.g. image with digit label Sensory data for car with intended steering control Data without labels: unsupervised learning E.g. automatic clustering (grouping) of sounds Clustering of text according to topics Data with and without labels: semi-supervised learning No examples: learning-by-doing Reinforcement learning Stefan Roth, Department of Computer Science GRIS 22

23 Some Key Challenges We need generalization! We cannot simply memorize the training set. What if we see an input that we haven t seen before? Different shape of the digit image (unknown writer) Dirt on the picture, etc. We need to learn what is important for carrying out our task. This is one of the most crucial points that we will return to many times. Stefan Roth, Department of Computer Science GRIS 23

24 Generalization How do we achieve generalization? width salmon sea bass 19 18? lightness FIGURE 1.5. Overly complex models for the fish will lead to decision boundaries that are complicated. While such a decision may lead to perfect classification of our training samples, it would lead to poor performance on future patterns. The novel test point marked? is evidently most likely a salmon, whereas the complex decision boundary shown leads it to be classified as a sea bass. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc. Stefan Roth, Department of Computer Science GRIS 24

25 Generalization How do we achieve generalization? width salmon sea bass Occam s Razor lightness FIGURE 1.6. The decision boundary shown might represent the optimal tradeoff between performance on the training set and simplicity of classifier, thereby giving the highest accuracy on new patterns. From: Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification. Copyright c 2001 by John Wiley & Sons, Inc. We should not make the model overly complex! Stefan Roth, Department of Computer Science GRIS 25

26 Some Key Challenges Input x: Features: Choosing the right features is very important. Coding and use of domain knowledge. May allow for invariance (e.g. volume and pitch of voice). Curse of Dimensionality: If the features are too high-dimensional, we will run into trouble - more later. Dimensionality reduction Stefan Roth, Department of Computer Science GRIS 26

27 Some Key Challenges How do we measure performance? 99% correct classification in speech recognition: What does that really mean? We understand the meaning of the sentence? We understand every word? For all speakers? Need more concrete numbers: % of correctly classified letters average distance driven (until accident...) % of games won % correctly recognized words, sentences, etc. Training vs. testing performance! Stefan Roth, Department of Computer Science GRIS 27

28 Some Key Challenges We also need to define the right error metric: Whis is better? Euclidean distance (L2 norm) might be useless. Stefan Roth, Department of Computer Science GRIS 28

29 Some Key Challenges Which is the right model? The learned parameters can mean a lot of different things. - w: may characterize the family of functions or the model space - w: may index the hypothesis space - w: vector, adjacency matrix, graph,... Stefan Roth, Department of Computer Science GRIS 29

30 Some Key Challenges Even if we have solved the other problems, computation is usually quite hard: Learning often involves some kind of optimization Find (search) best model parameters Often we have to deal with thousands of training examples Given a model, compute the prediction efficiently Stefan Roth, Department of Computer Science GRIS 30

31 Why is machine learning interesting (for you)? Machine learning is a challenging problem that is far from being solved. Our learning systems are primitive compared to us humans. Think about what and how quickly a child can learn! It combines insights and tools from many fields and disciplines: Traditional artificial intelligence (logic, semantic networks,...) Statistics Complexity theory Artificial neural networks Psychology Adaptive control,... Stefan Roth, Department of Computer Science GRIS 31

32 Why is machine learning interesting (for you)? Allows you to apply theoretical skills that you may otherwise only use rarely. Has lots of applications: Computer vision Computer linguistics Search (think Google) Digital assistants Computer systems... It is a growing field: Many major companies are hiring people with machine learning knowledge. Anecdote: At a recent workshop on computer graphics, about 2/3 of the groups said they would benefit from more machine learning knowledge. Stefan Roth, Department of Computer Science GRIS 32

33 Preliminary Syllabus Subject to change! Fundamentals (~ 3 weeks) Bayes decision theory, maximum likelihood, Bayesian inference Performance evaluation Probability density estimation Mixture models, expectation maximization Linear Methods (~ 3-4 weeks) Linear regression PCA, robust PCA Fisher linear discriminant generalized linear models Stefan Roth, Department of Computer Science GRIS 33

34 Preliminary Syllabus Large-Margin Methods (~ 3-4 weeks) Statistical learning theory Support vector machines Kernel methods Miscellaneous (~ 3 weeks) Model averaging (bagging & boosting) Graphical models (basic introduction) Stefan Roth, Department of Computer Science GRIS 34

35 Credits Large parts of the lecture material have been developed by Prof. Bernt Schiele for the previous iterations of this course. Many figures that I will use are directly taken out of the books by Chris Bishop and Duda, Hart & Stork. Stefan Roth, Department of Computer Science GRIS 35

36 Brief Review of Basic Probability What you should already know: F = a F = o Random picking: Red box: 60% of the time B = r B = b Blue box: 40% of the time Pick fruit from a box with equal probability p(b = r) = 0.6 p(b = b) = 0.4 p(f = a B = r) =p(f = o B = b) = 0.25 p(f = o B = r) =p(f = a B = b) = 0.75 Stefan Roth, Department of Computer Science GRIS 36

37 Brief Review of Basic Probability We usually do not mention the random variable (RV) explicitly (for brevity). Instead of p(x = x) we write: p(x) if we want to denote the probability distribution for a particular random variable X. p(x) if we want to denote the value of the probability of the random variable being x. It should be obvious from the context when we mean the random variable itself and a value that the random variable can take. Some people use upper case P (X = x) probability distributions. I usually don t. for (discrete) Stefan Roth, Department of Computer Science GRIS 37

38 Brief Review of Basic Probability Joint probability: The probability distribution of random variables X and Y taking on a configuration jointly. For example: Conditional probability: p(x, Y ) p(b = b, F = o) p(x Y ) The probability distribution of random variable X that random variable Y takes on a specific value For example: p(b = b F = o) given the fact Stefan Roth, Department of Computer Science GRIS 38

39 Basic Rules I Probabilities are always non-negative: p(x) 0 Probabilities sum to 1: p(x) = 1 0 p(x) 1 x Sum rule or marginalization: p(x) = y p(x, y) p(y) = x p(x, y) p(x) and p(y) are called marginal distributions of the joint distribution p(x, y) Stefan Roth, Department of Computer Science GRIS 39

40 Basic Rules II Product rule: p(x, y) =p(x y)p(y) =p(y x)p(x) From this we directly follow... Bayes rule or Bayes theorem: We will widely use these rules. p(y x) = p(x y)p(y) p(x) Rev. Thomas Bayes Stefan Roth, Department of Computer Science GRIS 40

41 Continuous RVs What if we have continuous random variables, X = x R say? Any single value has zero probability. We can only assign a probability for a random variable being in a range of values: Pr(x 0 < X < x 1 )=Pr(x 0 X x 1 ) Instead we use the probability density p(x) Pr(x 0 X x 1 )= x1 x 0 p(x) dx Cumulative distribution function: P (z) = z p(x) dx and P (x) =p(x) Stefan Roth, Department of Computer Science GRIS 41

42 Continuous RVs p(x) P (x) Probability density function = pdf δx x Cumulative distribution function = cdf We can work with a density (pdf) as if it was a probability distribution: For simplicity we usually use the same notation for both. Stefan Roth, Department of Computer Science GRIS 42

43 Basic rules for pdfs What are the rules? Non-negativity: Summing to 1: p(x) 0 p(x) dx =1 But: Marginalization: p(x) = in general p(x, y) dy p(y) = p(x, y) dx p(x) 1 Product rule: p(x, y) =p(x y)p(y) =p(y x)p(x) Stefan Roth, Department of Computer Science GRIS 43

44 Expectations The average value of a function f(x) distribution p(x) is the expectation: under a probability E[f] =E[f(x)] = x f(x)p(x) or E[f] = f(x)p(x) dx For joint distributions we sometimes write: E x [f(x, y)] Conditional expectation: E x y [f] =E x [f y] = x f(x)p(x y) Stefan Roth, Department of Computer Science GRIS 44

45 Variance and Covariance Variance of a single RV: var[x] =E [ (x E[x]) 2] = E[x 2 ] E[x] 2 Covariance of two RVs: cov(x, y) =E x,y [(x E[x])(y E[y])] = E x,y [xy] E[x]E[y] Random vectors: All we have said so far not only applies to scalar random variables, but also to random vectors. In particular, we have the covariance matrix: cov(x, y) = E x,y [ (x E[x])(y E[y]) T ] = E x,y [xy T ] E[x]E[y] T Stefan Roth, Department of Computer Science GRIS 45

46 Preview Review of some basics about probability Bayesian decision theory Loss functions Disclaimer: It will get quite a bit more mathematical than this :) Don t get scared away, but be aware that this will not be a walk in the park. Stefan Roth, Department of Computer Science GRIS 46

47 Readings for next week Introduction to ML: Bishop 1.0, 1.1 Review of the basics of probability: Bishop 1.2 (you can skip and for now) Decision theory: Bishop 1.5 For the curious: Probability: You could also look at MacKay 2 Brush up on information theory: Bishop 1.6 Stefan Roth, Department of Computer Science GRIS 47

203.4770: Introduction to Machine Learning Dr. Rita Osadchy

203.4770: Introduction to Machine Learning Dr. Rita Osadchy 203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:

More information

Introduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu

Introduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction to Machine Learning Lecture 1 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction Logistics Prerequisites: basics concepts needed in probability and statistics

More information

Machine Learning. 01 - Introduction

Machine Learning. 01 - Introduction Machine Learning 01 - Introduction Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge

More information

Introduction to Pattern Recognition

Introduction to Pattern Recognition Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)

More information

Learning is a very general term denoting the way in which agents:

Learning is a very general term denoting the way in which agents: What is learning? Learning is a very general term denoting the way in which agents: Acquire and organize knowledge (by building, modifying and organizing internal representations of some external reality);

More information

Network Machine Learning Research Group. Intended status: Informational October 19, 2015 Expires: April 21, 2016

Network Machine Learning Research Group. Intended status: Informational October 19, 2015 Expires: April 21, 2016 Network Machine Learning Research Group S. Jiang Internet-Draft Huawei Technologies Co., Ltd Intended status: Informational October 19, 2015 Expires: April 21, 2016 Abstract Network Machine Learning draft-jiang-nmlrg-network-machine-learning-00

More information

Machine Learning and Data Mining. Fundamentals, robotics, recognition

Machine Learning and Data Mining. Fundamentals, robotics, recognition Machine Learning and Data Mining Fundamentals, robotics, recognition Machine Learning, Data Mining, Knowledge Discovery in Data Bases Their mutual relations Data Mining, Knowledge Discovery in Databases,

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

MACHINE LEARNING IN HIGH ENERGY PHYSICS

MACHINE LEARNING IN HIGH ENERGY PHYSICS MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!

More information

MA2823: Foundations of Machine Learning

MA2823: Foundations of Machine Learning MA2823: Foundations of Machine Learning École Centrale Paris Fall 2015 Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe agathe.azencott@mines paristech.fr TAs: Jiaqian Yu jiaqian.yu@centralesupelec.fr

More information

Machine Learning CS 6830. Lecture 01. Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu

Machine Learning CS 6830. Lecture 01. Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Machine Learning CS 6830 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu What is Learning? Merriam-Webster: learn = to acquire knowledge, understanding, or skill

More information

Introduction to Machine Learning Using Python. Vikram Kamath

Introduction to Machine Learning Using Python. Vikram Kamath Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression

More information

CPSC 340: Machine Learning and Data Mining. Mark Schmidt University of British Columbia Fall 2015

CPSC 340: Machine Learning and Data Mining. Mark Schmidt University of British Columbia Fall 2015 CPSC 340: Machine Learning and Data Mining Mark Schmidt University of British Columbia Fall 2015 Outline 1) Intro to Machine Learning and Data Mining: Big data phenomenon and types of data. Definitions

More information

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not. Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

Machine Learning. Chapter 18, 21. Some material adopted from notes by Chuck Dyer

Machine Learning. Chapter 18, 21. Some material adopted from notes by Chuck Dyer Machine Learning Chapter 18, 21 Some material adopted from notes by Chuck Dyer What is learning? Learning denotes changes in a system that... enable a system to do the same task more efficiently the next

More information

Government of Russian Federation. Faculty of Computer Science School of Data Analysis and Artificial Intelligence

Government of Russian Federation. Faculty of Computer Science School of Data Analysis and Artificial Intelligence Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education National Research University «Higher School of Economics» Faculty of Computer Science School

More information

Machine Learning. CUNY Graduate Center, Spring 2013. Professor Liang Huang. huang@cs.qc.cuny.edu

Machine Learning. CUNY Graduate Center, Spring 2013. Professor Liang Huang. huang@cs.qc.cuny.edu Machine Learning CUNY Graduate Center, Spring 2013 Professor Liang Huang huang@cs.qc.cuny.edu http://acl.cs.qc.edu/~lhuang/teaching/machine-learning Logistics Lectures M 9:30-11:30 am Room 4419 Personnel

More information

Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density

Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,

More information

Question 2 Naïve Bayes (16 points)

Question 2 Naïve Bayes (16 points) Question 2 Naïve Bayes (16 points) About 2/3 of your email is spam so you downloaded an open source spam filter based on word occurrences that uses the Naive Bayes classifier. Assume you collected the

More information

Data Mining - Evaluation of Classifiers

Data Mining - Evaluation of Classifiers Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010

More information

Machine Learning and Pattern Recognition Logistic Regression

Machine Learning and Pattern Recognition Logistic Regression Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,

More information

Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)

Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision

More information

6.2.8 Neural networks for data mining

6.2.8 Neural networks for data mining 6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural

More information

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

More information

Maschinelles Lernen mit MATLAB

Maschinelles Lernen mit MATLAB Maschinelles Lernen mit MATLAB Jérémy Huard Applikationsingenieur The MathWorks GmbH 2015 The MathWorks, Inc. 1 Machine Learning is Everywhere Image Recognition Speech Recognition Stock Prediction Medical

More information

Machine Learning and Statistics: What s the Connection?

Machine Learning and Statistics: What s the Connection? Machine Learning and Statistics: What s the Connection? Institute for Adaptive and Neural Computation School of Informatics, University of Edinburgh, UK August 2006 Outline The roots of machine learning

More information

KATE GLEASON COLLEGE OF ENGINEERING. John D. Hromi Center for Quality and Applied Statistics

KATE GLEASON COLLEGE OF ENGINEERING. John D. Hromi Center for Quality and Applied Statistics ROCHESTER INSTITUTE OF TECHNOLOGY COURSE OUTLINE FORM KATE GLEASON COLLEGE OF ENGINEERING John D. Hromi Center for Quality and Applied Statistics NEW (or REVISED) COURSE (KGCOE- CQAS- 747- Principles of

More information

Faculty of Science School of Mathematics and Statistics

Faculty of Science School of Mathematics and Statistics Faculty of Science School of Mathematics and Statistics MATH5836 Data Mining and its Business Applications Semester 1, 2014 CRICOS Provider No: 00098G MATH5836 Course Outline Information about the course

More information

Kickoff: Anomaly Detection Challenges

Kickoff: Anomaly Detection Challenges Kickoff: Anomaly Detection Challenges A Practical Course in SS2014 Huang Xiao Han Xiao Chair of IT Security (I20) Department of Informatics Technische Universität München January 31, 2014 Huang Xiao, Han

More information

Classification Techniques for Remote Sensing

Classification Techniques for Remote Sensing Classification Techniques for Remote Sensing Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara saksoy@cs.bilkent.edu.tr http://www.cs.bilkent.edu.tr/ saksoy/courses/cs551

More information

Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches

Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Machine Learning Capacity and Performance Analysis and R

Machine Learning Capacity and Performance Analysis and R Machine Learning and R May 3, 11 30 25 15 10 5 25 15 10 5 30 25 15 10 5 0 2 4 6 8 101214161822 0 2 4 6 8 101214161822 0 2 4 6 8 101214161822 100 80 60 40 100 80 60 40 100 80 60 40 30 25 15 10 5 25 15 10

More information

Machine Learning. Mausam (based on slides by Tom Mitchell, Oren Etzioni and Pedro Domingos)

Machine Learning. Mausam (based on slides by Tom Mitchell, Oren Etzioni and Pedro Domingos) Machine Learning Mausam (based on slides by Tom Mitchell, Oren Etzioni and Pedro Domingos) What Is Machine Learning? A computer program is said to learn from experience E with respect to some class of

More information

CSE 517A MACHINE LEARNING INTRODUCTION

CSE 517A MACHINE LEARNING INTRODUCTION CSE 517A MACHINE LEARNING INTRODUCTION Spring 2016 Marion Neumann Contents in these slides may be subject to copyright. Some materials are adopted from Killian Weinberger. Thanks, Killian! Machine Learning

More information

Lecture Slides for INTRODUCTION TO. ETHEM ALPAYDIN The MIT Press, 2004. Lab Class and literature. Friday, 9.00 10.00, Harburger Schloßstr.

Lecture Slides for INTRODUCTION TO. ETHEM ALPAYDIN The MIT Press, 2004. Lab Class and literature. Friday, 9.00 10.00, Harburger Schloßstr. Lecture Slides for INTRODUCTION TO Machine Learning ETHEM ALPAYDIN The MIT Press, 2004 alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml Lab Class and literature Friday, 9.00 10.00, Harburger

More information

Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing

Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and

More information

T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier. Santosh Tirunagari : 245577

T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier. Santosh Tirunagari : 245577 T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier Santosh Tirunagari : 245577 January 20, 2011 Abstract This term project gives a solution how to classify an email as spam or

More information

TIETS34 Seminar: Data Mining on Biometric identification

TIETS34 Seminar: Data Mining on Biometric identification TIETS34 Seminar: Data Mining on Biometric identification Youming Zhang Computer Science, School of Information Sciences, 33014 University of Tampere, Finland Youming.Zhang@uta.fi Course Description Content

More information

CSCI567 Machine Learning (Fall 2014)

CSCI567 Machine Learning (Fall 2014) CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /

More information

MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr

MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Department of Applied Mathematics Ecole Centrale Paris Galen

More information

8. Machine Learning Applied Artificial Intelligence

8. Machine Learning Applied Artificial Intelligence 8. Machine Learning Applied Artificial Intelligence Prof. Dr. Bernhard Humm Faculty of Computer Science Hochschule Darmstadt University of Applied Sciences 1 Retrospective Natural Language Processing Name

More information

Machine learning for algo trading

Machine learning for algo trading Machine learning for algo trading An introduction for nonmathematicians Dr. Aly Kassam Overview High level introduction to machine learning A machine learning bestiary What has all this got to do with

More information

Supervised Learning (Big Data Analytics)

Supervised Learning (Big Data Analytics) Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used

More information

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

More information

HT2015: SC4 Statistical Data Mining and Machine Learning

HT2015: SC4 Statistical Data Mining and Machine Learning HT2015: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Bayesian Nonparametrics Parametric vs Nonparametric

More information

Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

More information

Analysis Tools and Libraries for BigData

Analysis Tools and Libraries for BigData + Analysis Tools and Libraries for BigData Lecture 02 Abhijit Bendale + Office Hours 2 n Terry Boult (Waiting to Confirm) n Abhijit Bendale (Tue 2:45 to 4:45 pm). Best if you email me in advance, but I

More information

An Overview of Knowledge Discovery Database and Data mining Techniques

An Overview of Knowledge Discovery Database and Data mining Techniques An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,

More information

1. Let A, B and C are three events such that P(A) = 0.45, P(B) = 0.30, P(C) = 0.35,

1. Let A, B and C are three events such that P(A) = 0.45, P(B) = 0.30, P(C) = 0.35, 1. Let A, B and C are three events such that PA =.4, PB =.3, PC =.3, P A B =.6, P A C =.6, P B C =., P A B C =.7. a Compute P A B, P A C, P B C. b Compute P A B C. c Compute the probability that exactly

More information

Machine Learning in Spam Filtering

Machine Learning in Spam Filtering Machine Learning in Spam Filtering A Crash Course in ML Konstantin Tretyakov kt@ut.ee Institute of Computer Science, University of Tartu Overview Spam is Evil ML for Spam Filtering: General Idea, Problems.

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

MACHINE LEARNING BASICS WITH R

MACHINE LEARNING BASICS WITH R MACHINE LEARNING [Hands-on Introduction of Supervised Machine Learning Methods] DURATION 2 DAY The field of machine learning is concerned with the question of how to construct computer programs that automatically

More information

Machine Learning with MATLAB David Willingham Application Engineer

Machine Learning with MATLAB David Willingham Application Engineer Machine Learning with MATLAB David Willingham Application Engineer 2014 The MathWorks, Inc. 1 Goals Overview of machine learning Machine learning models & techniques available in MATLAB Streamlining the

More information

Outlier Ensembles. Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598. Keynote, Outlier Detection and Description Workshop, 2013

Outlier Ensembles. Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598. Keynote, Outlier Detection and Description Workshop, 2013 Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier

More information

Using Data Mining for Mobile Communication Clustering and Characterization

Using Data Mining for Mobile Communication Clustering and Characterization Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer

More information

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering Martin

More information

Classification Problems

Classification Problems Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems

More information

Learning outcomes. Knowledge and understanding. Competence and skills

Learning outcomes. Knowledge and understanding. Competence and skills Syllabus Master s Programme in Statistics and Data Mining 120 ECTS Credits Aim The rapid growth of databases provides scientists and business people with vast new resources. This programme meets the challenges

More information

An Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015

An Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015 An Introduction to Data Mining for Wind Power Management Spring 2015 Big Data World Every minute: Google receives over 4 million search queries Facebook users share almost 2.5 million pieces of content

More information

INTRODUCTION TO MACHINE LEARNING 3RD EDITION

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN The MIT Press, 2014 Lecture Slides for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml3e CHAPTER 1: INTRODUCTION Big Data 3 Widespread

More information

An Introduction to Data Mining

An Introduction to Data Mining An Introduction to Intel Beijing wei.heng@intel.com January 17, 2014 Outline 1 DW Overview What is Notable Application of Conference, Software and Applications Major Process in 2 Major Tasks in Detail

More information

Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data

Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data CMPE 59H Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Non-linear

More information

Neural Networks and Back Propagation Algorithm

Neural Networks and Back Propagation Algorithm Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important

More information

Machine Learning: Overview

Machine Learning: Overview Machine Learning: Overview Why Learning? Learning is a core of property of being intelligent. Hence Machine learning is a core subarea of Artificial Intelligence. There is a need for programs to behave

More information

Information Management course

Information Management course Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)

More information

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4-day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Lecture 6. Artificial Neural Networks

Lecture 6. Artificial Neural Networks Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

More information

01219211 Software Development Training Camp 1 (0-3) Prerequisite : 01204214 Program development skill enhancement camp, at least 48 person-hours.

01219211 Software Development Training Camp 1 (0-3) Prerequisite : 01204214 Program development skill enhancement camp, at least 48 person-hours. (International Program) 01219141 Object-Oriented Modeling and Programming 3 (3-0) Object concepts, object-oriented design and analysis, object-oriented analysis relating to developing conceptual models

More information

Machine Learning for Data Science (CS4786) Lecture 1

Machine Learning for Data Science (CS4786) Lecture 1 Machine Learning for Data Science (CS4786) Lecture 1 Tu-Th 10:10 to 11:25 AM Hollister B14 Instructors : Lillian Lee and Karthik Sridharan ROUGH DETAILS ABOUT THE COURSE Diagnostic assignment 0 is out:

More information

New Ensemble Combination Scheme

New Ensemble Combination Scheme New Ensemble Combination Scheme Namhyoung Kim, Youngdoo Son, and Jaewook Lee, Member, IEEE Abstract Recently many statistical learning techniques are successfully developed and used in several areas However,

More information

STT315 Chapter 4 Random Variables & Probability Distributions KM. Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables

STT315 Chapter 4 Random Variables & Probability Distributions KM. Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables Discrete vs. continuous random variables Examples of continuous distributions o Uniform o Exponential o Normal Recall: A random

More information

Exploratory Data Analysis with MATLAB

Exploratory Data Analysis with MATLAB Computer Science and Data Analysis Series Exploratory Data Analysis with MATLAB Second Edition Wendy L Martinez Angel R. Martinez Jeffrey L. Solka ( r ec) CRC Press VV J Taylor & Francis Group Boca Raton

More information

Emotion Detection from Speech

Emotion Detection from Speech Emotion Detection from Speech 1. Introduction Although emotion detection from speech is a relatively new field of research, it has many potential applications. In human-computer or human-human interaction

More information

Chapter 6. The stacking ensemble approach

Chapter 6. The stacking ensemble approach 82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described

More information

Neural Networks Lesson 5 - Cluster Analysis

Neural Networks Lesson 5 - Cluster Analysis Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it Rome, 29

More information

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Web-based Analytics Table

More information

An Introduction to Machine Learning

An Introduction to Machine Learning An Introduction to Machine Learning L5: Novelty Detection and Regression Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune,

More information

Analytics on Big Data

Analytics on Big Data Analytics on Big Data Riccardo Torlone Università Roma Tre Credits: Mohamed Eltabakh (WPI) Analytics The discovery and communication of meaningful patterns in data (Wikipedia) It relies on data analysis

More information

The Chinese Restaurant Process

The Chinese Restaurant Process COS 597C: Bayesian nonparametrics Lecturer: David Blei Lecture # 1 Scribes: Peter Frazier, Indraneel Mukherjee September 21, 2007 In this first lecture, we begin by introducing the Chinese Restaurant Process.

More information

Support Vector Machine. Tutorial. (and Statistical Learning Theory)

Support Vector Machine. Tutorial. (and Statistical Learning Theory) Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. jasonw@nec-labs.com 1 Support Vector Machines: history SVMs introduced

More information

Statistical Validation and Data Analytics in ediscovery. Jesse Kornblum

Statistical Validation and Data Analytics in ediscovery. Jesse Kornblum Statistical Validation and Data Analytics in ediscovery Jesse Kornblum Administrivia Silence your mobile Interactive talk Please ask questions 2 Outline Introduction Big Questions What Makes Things Similar?

More information

Fall 2012 Q530. Programming for Cognitive Science

Fall 2012 Q530. Programming for Cognitive Science Fall 2012 Q530 Programming for Cognitive Science Aimed at little or no programming experience. Improve your confidence and skills at: Writing code. Reading code. Understand the abilities and limitations

More information

Java Modules for Time Series Analysis

Java Modules for Time Series Analysis Java Modules for Time Series Analysis Agenda Clustering Non-normal distributions Multifactor modeling Implied ratings Time series prediction 1. Clustering + Cluster 1 Synthetic Clustering + Time series

More information

Machine Learning using MapReduce

Machine Learning using MapReduce Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

More information

Support Vector Machines with Clustering for Training with Very Large Datasets

Support Vector Machines with Clustering for Training with Very Large Datasets Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano

More information

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

More information

Statistics W4240: Data Mining Columbia University Spring, 2014

Statistics W4240: Data Mining Columbia University Spring, 2014 Statistics W4240: Data Mining Columbia University Spring, 2014 Version: January 30, 2014. The syllabus is subject to change, so look for the version with the most recent date. Course Description Massive

More information

Classification algorithm in Data mining: An Overview

Classification algorithm in Data mining: An Overview Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department

More information

COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK DEPARTMENT OF INDUSTRIAL ENGINEERING AND OPERATIONS RESEARCH

COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK DEPARTMENT OF INDUSTRIAL ENGINEERING AND OPERATIONS RESEARCH Course: IEOR 4575 Business Analytics for Operations Research Lectures MW 2:40-3:55PM Instructor Prof. Guillermo Gallego Office Hours Tuesdays: 3-4pm Office: CEPSR 822 (8 th floor) Textbooks and Learning

More information

Machine Learning Logistic Regression

Machine Learning Logistic Regression Machine Learning Logistic Regression Jeff Howbert Introduction to Machine Learning Winter 2012 1 Logistic regression Name is somewhat misleading. Really a technique for classification, not regression.

More information

1 Maximum likelihood estimation

1 Maximum likelihood estimation COS 424: Interacting with Data Lecturer: David Blei Lecture #4 Scribes: Wei Ho, Michael Ye February 14, 2008 1 Maximum likelihood estimation 1.1 MLE of a Bernoulli random variable (coin flips) Given N

More information

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

More information

Machine Learning Big Data using Map Reduce

Machine Learning Big Data using Map Reduce Machine Learning Big Data using Map Reduce By Michael Bowles, PhD Where Does Big Data Come From? -Web data (web logs, click histories) -e-commerce applications (purchase histories) -Retail purchase histories

More information

CSci 538 Articial Intelligence (Machine Learning and Data Analysis)

CSci 538 Articial Intelligence (Machine Learning and Data Analysis) CSci 538 Articial Intelligence (Machine Learning and Data Analysis) Course Syllabus Fall 2015 Instructor Derek Harter, Ph.D., Associate Professor Department of Computer Science Texas A&M University - Commerce

More information

MS1b Statistical Data Mining

MS1b Statistical Data Mining MS1b Statistical Data Mining Yee Whye Teh Department of Statistics Oxford http://www.stats.ox.ac.uk/~teh/datamining.html Outline Administrivia and Introduction Course Structure Syllabus Introduction to

More information