# Machine Learning. CS 188: Artificial Intelligence Naïve Bayes. Example: Digit Recognition. Other Classification Tasks

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 CS 188: Artificial Intelligence Naïve Bayes Machine Learning Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters (e.g. probabilities) Learning structure (e.g. BN graphs) Learning hidden concepts (e.g. clustering) Today: model-based classification with Naive Bayes Dan Klein, Pieter Abbeel University of California, Berkeley Classification Example: Spam Filter Input: an Output: spam/ham Setup: Get a large collection of example s, each labeled spam or ham Note: someone has to hand label all this data! Want to learn to predict labels of new, future s Features: The attributes used to make the ham / spam decision Words: FREE! Text Patterns: \$dd, CAPS Non-text: SenderInContacts Dear Sir. First, I must solicit your confidence in this transaction, this is by virture of its nature as being utterly confidencial and top secret. TO BE REMOVED FROM FUTURE MAILINGS, SIMPLY REPLY TO THIS MESSAGE AND PUT "REMOVE" IN THE SUBJECT. 99 MILLION ADDRESSES FOR ONLY \$99 Ok, Iknow this is blatantly OT but I'm beginning to go insane. Had an old Dell Dimension XPS sitting in the corner and decided to put it to use, I know it was working pre being stuck in the corner, but when I plugged it in, hit the power nothing happened. Input: images / pixel grids Output: a digit 0-9 Example: Digit Recognition Setup: Get a large collection of example images, each labeled with a digit Note: someone has to hand label all this data! Want to learn to predict labels of new, future digit images Features: The attributes used to make the digit decision Pixels: (6,8)=ON Shape Patterns: NumComponents, AspectRatio, NumLoops ?? Other Classification Tasks Classification: given inputs x, predict labels (classes) y Examples: Spam detection (input: document, classes: spam / ham) OCR (input: images, classes: characters) Medical diagnosis (input: symptoms, classes: diseases) Automatic essay grading (input: document, classes: grades) Fraud detection (input: account activity, classes: fraud / no fraud) Customer service routing many more Classification is an important commercial technology! 1

2 Model-Based Classification Model-Based Classification Model-based approach Build a model (e.g. Bayesnet) where both the label and features are random variables Instantiate any observed features Query for the distribution of the label conditioned on the features Challenges What structure should the BN have? How should we learn its parameters? Naïve Bayes for Digits General Naïve Bayes Naïve Bayes: Assume all features are independent effects of the label Simple digit recognition version: One feature (variable) F ij for each grid position <i,j> Feature values are on / off, based on whether intensity is more or less than 0.5 in underlying image Each input maps to a feature vector, e.g. Here: lots of features, each is binary valued F 1 F 2 Y F n A general Naive Bayesmodel: Y x F n values Y parameters n x F x Y parameters F 1 F 2 Y F n Naïve Bayes model: What do we need to learn? We only have to specify how each feature depends on the class Total number of parameters is linearin n Model is very simplistic, but often works anyway Inference for Naïve Bayes General Naïve Bayes Goal: compute posterior distribution over label variable Y Step 1: get joint probability of label and evidence for each label What do we need in order to use Naïve Bayes? Inference method (we just saw this part) Start with a bunch of probabilities: P(Y) and the P(F i Y) tables Use standard inference to compute P(Y F 1 F n ) Nothing new here Step 2: sum to get probability of evidence Step 3: normalize by dividing Step 1 by Step 2 + Estimates of local conditional probability tables P(Y), the prior over labels P(F i Y) for each feature (evidence variable) These probabilities are collectively called the parametersof the model and denoted by θ Up until now, we assumed these appeared by magic, but they typically come from training data counts: we ll look at this soon 2

3 Example: Conditional Probabilities Naïve Bayes for Text Bag-of-words Naïve Bayes: Features: W i is the word at positoni As before: predict label conditioned on feature variables (spam vs. ham) As before: assume features are conditionally independent given label New: each W i is identically distributed Word at position i, not i th word in the dictionary! Generative model: Tied distributions and bag-of-words Usually, each variable gets its own conditional probability distribution P(F Y) In a bag-of-words model Each position is identically distributed All positions share the same conditional probsp(w Y) Why make this assumption? Called bag-of-words because model is insensitive to word order or reordering Example: Spam Filtering Spam Example Model: What are the parameters? ham : 0.66 spam: 0.33 Where do these tables come from? the : to : and : of : you : a : with: from: the : to : of : : with: from: and : a : Word P(w spam) P(w ham) Tot Spam Tot Ham (prior) Gary would you like to lose weight while you sleep P(spam w) = 98.9 Training and Testing Important Concepts Data: labeled instances, e.g. s marked spam/ham Training set Held out set Test set Features: attribute-value pairs which characterize each x Experimentation cycle Learn parameters (e.g. model probabilities) on training set (Tune hyperparameterson held-out set) Compute accuracy of test set Very important: never peek at the test set! Evaluation Accuracy: fraction of instances predicted correctly Overfittingand generalization Want a classifier which does well on testdata Overfitting: fitting the training data very closely, but not generalizing well We ll investigate overfittingand generalization formally in a few lectures Training Data Held-Out Data Test Data 3

4 Generalization and Overfitting Overfitting Degree 15 polynomial Example: Overfitting Example: Overfitting Posteriors determined by relative probabilities (odds ratios): south-west : inf nation : inf morally : inf nicely : inf extent : inf seriously : inf screens : inf minute : inf guaranteed : inf \$ : inf delivery : inf signature : inf 2 wins!! What went wrong here? Generalization and Overfitting Parameter Estimation Relative frequency parameters will overfit the training data! Just because we never saw a 3 with pixel (15,15) on during training doesn t mean we won t see it at test time Unlikely that every occurrence of minute is 100% spam Unlikely that every occurrence of seriously is 100% ham What about all the words that don t occur in the training set at all? In general, we can t go around giving unseen events zero probability As an extreme case, imagine using the entire as the only feature Would get the training data perfect (if deterministic labeling) Wouldn t generalizeat all Just making the bag-of-words assumption gives us some generalization, but isn t enough To generalize better: we need to smooth or regularize the estimates 4

5 Parameter Estimation Smoothing Estimating the distribution of a random variable Elicitation:ask a human (why is this hard?) Empirically: use training data (learning!) E.g.: for each outcome x, look at the empirical rateof that value: r b b r b b r b b r r b b b b r r b This is the estimate that maximizes the likelihood of the data Maximum Likelihood? Unseen Events Relative frequencies are the maximum likelihood estimates Another option is to consider the most likely parameter value given the data???? Laplace Smoothing Laplace Smoothing Laplace s estimate: Pretend you saw every outcome once more than you actually did r r b Laplace s estimate (extended): Pretend you saw every outcome k extra times r r b What s Laplace with k = 0? k is the strengthof the prior Can derive this estimate with Dirichletpriors(see cs281a) Laplace for conditionals: Smooth each condition independently: 5

7 What to Do About Errors? Need more features words aren t enough! Have you ed the sender before? Have 1K other people just gotten the same ? Is the sending information consistent? Is the in ALL CAPS? Do inline URLs point where they say they point? Does the address you by (your) name? Can add these information sources as new variables in the NB model Next class we ll talk about classifiers which let you easily add arbitrary features more easily 7

### Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing

CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and

### CSE 473: Artificial Intelligence Autumn 2010

CSE 473: Artificial Intelligence Autumn 2010 Machine Learning: Naive Bayes and Perceptron Luke Zettlemoyer Many slides over the course adapted from Dan Klein. 1 Outline Learning: Naive Bayes and Perceptron

### Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)

Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision

### 1 Maximum likelihood estimation

COS 424: Interacting with Data Lecturer: David Blei Lecture #4 Scribes: Wei Ho, Michael Ye February 14, 2008 1 Maximum likelihood estimation 1.1 MLE of a Bernoulli random variable (coin flips) Given N

### Bayes and Naïve Bayes. cs534-machine Learning

Bayes and aïve Bayes cs534-machine Learning Bayes Classifier Generative model learns Prediction is made by and where This is often referred to as the Bayes Classifier, because of the use of the Bayes rule

### Question 2 Naïve Bayes (16 points)

Question 2 Naïve Bayes (16 points) About 2/3 of your email is spam so you downloaded an open source spam filter based on word occurrences that uses the Naive Bayes classifier. Assume you collected the

### Reasoning Component Architecture

Architecture of a Spam Filter Application By Avi Pfeffer A spam filter consists of two components. In this article, based on my book Practical Probabilistic Programming, first describe the architecture

### On Attacking Statistical Spam Filters

On Attacking Statistical Spam Filters Gregory L. Wittel and S. Felix Wu Department of Computer Science University of California, Davis One Shields Avenue, Davis, CA 95616 USA Paper review by Deepak Chinavle

### Introduction to Bayesian Classification (A Practical Discussion) Todd Holloway Lecture for B551 Nov. 27, 2007

Introduction to Bayesian Classification (A Practical Discussion) Todd Holloway Lecture for B551 Nov. 27, 2007 Naïve Bayes Components ML vs. MAP Benefits Feature Preparation Filtering Decay Extended Examples

### Learning from Data: Naive Bayes

Semester 1 http://www.anc.ed.ac.uk/ amos/lfd/ Naive Bayes Typical example: Bayesian Spam Filter. Naive means naive. Bayesian methods can be much more sophisticated. Basic assumption: conditional independence.

### CS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.

Lecture Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott

### Defending Networks with Incomplete Information: A Machine Learning Approach. Alexandre Pinto alexcp@mlsecproject.org @alexcpsec @MLSecProject

Defending Networks with Incomplete Information: A Machine Learning Approach Alexandre Pinto alexcp@mlsecproject.org @alexcpsec @MLSecProject Agenda Security Monitoring: We are doing it wrong Machine Learning

### Creating a NL Texas Hold em Bot

Creating a NL Texas Hold em Bot Introduction Poker is an easy game to learn by very tough to master. One of the things that is hard to do is controlling emotions. Due to frustration, many have made the

### Machine Learning. CUNY Graduate Center, Spring 2013. Professor Liang Huang. huang@cs.qc.cuny.edu

Machine Learning CUNY Graduate Center, Spring 2013 Professor Liang Huang huang@cs.qc.cuny.edu http://acl.cs.qc.edu/~lhuang/teaching/machine-learning Logistics Lectures M 9:30-11:30 am Room 4419 Personnel

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Economics 1011a: Intermediate Microeconomics

Lecture 12: More Uncertainty Economics 1011a: Intermediate Microeconomics Lecture 12: More on Uncertainty Thursday, October 23, 2008 Last class we introduced choice under uncertainty. Today we will explore

### Classification Problems

Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems

### Supervised and unsupervised learning - 1

Chapter 3 Supervised and unsupervised learning - 1 3.1 Introduction The science of learning plays a key role in the field of statistics, data mining, artificial intelligence, intersecting with areas in

### BLOSSOMS MODULE ARE RANDOM TRIANGLES ACUTE OR OBTUSE? By Gilbert Strang Professor of Mathematics Massachusetts Institute of Technology

BLOSSOMS MODULE ARE RANDOM TRIANGLES ACUTE OR OBTUSE? By Gilbert Strang Professor of Mathematics Massachusetts Institute of Technology Hi! I m Gilbert Strang. I teach math at MIT. Mostly what I teach is

### Anti-Spam Filter Based on Naïve Bayes, SVM, and KNN model

AI TERM PROJECT GROUP 14 1 Anti-Spam Filter Based on,, and model Yun-Nung Chen, Che-An Lu, Chao-Yu Huang Abstract spam email filters are a well-known and powerful type of filters. We construct different

### Supervised Learning (Big Data Analytics)

Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used

### CSCI567 Machine Learning (Fall 2014)

CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /

### Discrete Structures for Computer Science

Discrete Structures for Computer Science Adam J. Lee adamlee@cs.pitt.edu 6111 Sennott Square Lecture #20: Bayes Theorem November 5, 2013 How can we incorporate prior knowledge? Sometimes we want to know

### Lecture 2: Introduction to belief (Bayesian) networks

Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (I-maps) January 7, 2008 1 COMP-526 Lecture 2 Recall from last time: Conditional

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Email Marketing 10Mistakes

Most Common Email Marketing 10Mistakes At Upper Case, we see very smart customers make mistakes that cause their email response rates to suffer. Here are the most common mistakes we encounter... 01 Not

### Machine Learning CS 6830. Lecture 01. Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu

Machine Learning CS 6830 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu What is Learning? Merriam-Webster: learn = to acquire knowledge, understanding, or skill

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Performance Analysis of Naive Bayes and J48 Classification Algorithm for Data Classification

Performance Analysis of Naive Bayes and J48 Classification Algorithm for Data Classification Tina R. Patil, Mrs. S. S. Sherekar Sant Gadgebaba Amravati University, Amravati tnpatil2@gmail.com, ss_sherekar@rediffmail.com

### Homework 4 Statistics W4240: Data Mining Columbia University Due Tuesday, October 29 in Class

Problem 1. (10 Points) James 6.1 Problem 2. (10 Points) James 6.3 Problem 3. (10 Points) James 6.5 Problem 4. (15 Points) James 6.7 Problem 5. (15 Points) James 6.10 Homework 4 Statistics W4240: Data Mining

### Christfried Webers. Canberra February June 2015

c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

### Simple Language Models for Spam Detection

Simple Language Models for Spam Detection Egidio Terra Faculty of Informatics PUC/RS - Brazil Abstract For this year s Spam track we used classifiers based on language models. These models are used to

### Machine Learning using MapReduce

Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

### CS 188: Artificial Intelligence. Probability recap

CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability

### Gaussian Classifiers CS498

Gaussian Classifiers CS498 Today s lecture The Gaussian Gaussian classifiers A slightly more sophisticated classifier Nearest Neighbors We can classify with nearest neighbors x m 1 m 2 Decision boundary

### Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Models vs. Patterns Models A model is a high level, global description of a

### Search Taxonomy. Web Search. Search Engine Optimization. Information Retrieval

Information Retrieval INFO 4300 / CS 4300! Retrieval models Older models» Boolean retrieval» Vector Space model Probabilistic Models» BM25» Language models Web search» Learning to Rank Search Taxonomy!

### 1 What is Machine Learning?

COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #1 Scribe: Rob Schapire February 4, 2008 1 What is Machine Learning? Machine learning studies computer algorithms for learning to do

### 203.4770: Introduction to Machine Learning Dr. Rita Osadchy

203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### Big Data Analytics CSCI 4030

High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Web advertising

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### Machine Learning Final Project Spam Email Filtering

Machine Learning Final Project Spam Email Filtering March 2013 Shahar Yifrah Guy Lev Table of Content 1. OVERVIEW... 3 2. DATASET... 3 2.1 SOURCE... 3 2.2 CREATION OF TRAINING AND TEST SETS... 4 2.3 FEATURE

### CS 2750 Machine Learning. Lecture 1. Machine Learning. CS 2750 Machine Learning.

Lecture 1 Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x-5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott

### Tweaking Naïve Bayes classifier for intelligent spam detection

682 Tweaking Naïve Bayes classifier for intelligent spam detection Ankita Raturi 1 and Sunil Pranit Lal 2 1 University of California, Irvine, CA 92697, USA. araturi@uci.edu 2 School of Computing, Information

### 6.2.8 Neural networks for data mining

6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural

### Logistic Regression for Spam Filtering

Logistic Regression for Spam Filtering Nikhila Arkalgud February 14, 28 Abstract The goal of the spam filtering problem is to identify an email as a spam or not spam. One of the classic techniques used

### Programming Exercise 3: Multi-class Classification and Neural Networks

Programming Exercise 3: Multi-class Classification and Neural Networks Machine Learning November 4, 2011 Introduction In this exercise, you will implement one-vs-all logistic regression and neural networks

### Neural Networks for Machine Learning. Lecture 13a The ups and downs of backpropagation

Neural Networks for Machine Learning Lecture 13a The ups and downs of backpropagation Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed A brief history of backpropagation

### T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier. Santosh Tirunagari : 245577

T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier Santosh Tirunagari : 245577 January 20, 2011 Abstract This term project gives a solution how to classify an email as spam or

### large-scale machine learning revisited Léon Bottou Microsoft Research (NYC)

large-scale machine learning revisited Léon Bottou Microsoft Research (NYC) 1 three frequent ideas in machine learning. independent and identically distributed data This experimental paradigm has driven

### Linear Classification. Volker Tresp Summer 2015

Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

### Data Mining - Evaluation of Classifiers

Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010

### Class Overview and General Introduction to Machine Learning

Class Overview and General Introduction to Machine Learning Piyush Rai www.cs.utah.edu/~piyush CS5350/6350: Machine Learning August 23, 2011 (CS5350/6350) Intro to ML August 23, 2011 1 / 25 Course Logistics

### An Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015

An Introduction to Data Mining for Wind Power Management Spring 2015 Big Data World Every minute: Google receives over 4 million search queries Facebook users share almost 2.5 million pieces of content

### Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007

Neural Networks Introduction to Artificial Intelligence CSE 150 May 29, 2007 Administration Last programming assignment has been posted! Final Exam: Tuesday, June 12, 11:30-2:30 Last Lecture Naïve Bayes

### Club Accounts. 2011 Question 6.

Club Accounts. 2011 Question 6. Anyone familiar with Farm Accounts or Service Firms (notes for both topics are back on the webpage you found this on), will have no trouble with Club Accounts. Essentially

### Probabilistic user behavior models in online stores for recommender systems

Probabilistic user behavior models in online stores for recommender systems Tomoharu Iwata Abstract Recommender systems are widely used in online stores because they are expected to improve both user

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### University of Colorado Boulder Colorado Springs Denver Anschutz Medical Campus. CU ecomm Program Email Marketing Best Practices

University of Colorado Boulder Colorado Springs Denver Anschutz Medical Campus CU ecomm Program Email Marketing Best Practices What is ecomm?... 2 Why use ecomm?... 3 Legal Considerations... 4 Email Marketing

### Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

### Making Sense of the Mayhem: Machine Learning and March Madness

Making Sense of the Mayhem: Machine Learning and March Madness Alex Tran and Adam Ginzberg Stanford University atran3@stanford.edu ginzberg@stanford.edu I. Introduction III. Model The goal of our research

### HT2015: SC4 Statistical Data Mining and Machine Learning

HT2015: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Bayesian Nonparametrics Parametric vs Nonparametric

Facebook Friend Suggestion Eytan Daniyalzade and Tim Lipus 1. Introduction Facebook is a social networking website with an open platform that enables developers to extract and utilize user information

### Evaluation in Machine Learning (2) Aims. Introduction

Acknowledgements Evaluation in Machine Learning (2) 15s1: COMP9417 Machine Learning and Data Mining School of Computer Science and Engineering, University of New South Wales Material derived from slides

### Log-Linear Models a.k.a. Logistic Regression, Maximum Entropy Models

Log-Linear Models a.k.a. Logistic Regression, Maximum Entropy Models Natural Language Processing CS 6120 Spring 2014 Northeastern University David Smith (some slides from Jason Eisner and Dan Klein) summary

### 1 Introductory Comments. 2 Bayesian Probability

Introductory Comments First, I would like to point out that I got this material from two sources: The first was a page from Paul Graham s website at www.paulgraham.com/ffb.html, and the second was a paper

### Sentiment analysis using emoticons

Sentiment analysis using emoticons Royden Kayhan Lewis Moharreri Steven Royden Ware Lewis Kayhan Steven Moharreri Ware Department of Computer Science, Ohio State University Problem definition Our aim was

### Statistical Validation and Data Analytics in ediscovery. Jesse Kornblum

Statistical Validation and Data Analytics in ediscovery Jesse Kornblum Administrivia Silence your mobile Interactive talk Please ask questions 2 Outline Introduction Big Questions What Makes Things Similar?

### Spam Filtering based on Naive Bayes Classification. Tianhao Sun

Spam Filtering based on Naive Bayes Classification Tianhao Sun May 1, 2009 Abstract This project discusses about the popular statistical spam filtering process: naive Bayes classification. A fairly famous

### Introduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu

Introduction to Machine Learning Lecture 1 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction Logistics Prerequisites: basics concepts needed in probability and statistics

### Object Recognition. Selim Aksoy. Bilkent University saksoy@cs.bilkent.edu.tr

Image Classification and Object Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Image classification Image (scene) classification is a fundamental

### A6- Sensitive Data Exposure

OWASP Vulnerabilities and Attacks Simplifie d: Business Manager Series Part 2 Have you heard of the times when Fantastic Frank from Randomland was furious? Money and critical data was being stolen from

### THE OPTIMIZER HANDBOOK:

THE OPTIMIZER HANDBOOK: LEAD SCORING Section 1: What is Lead Scoring? Picture this: you re selling vehicles, and you have a list that features over two hundred different leads, but you re only allowed

### Towards better accuracy for Spam predictions

Towards better accuracy for Spam predictions Chengyan Zhao Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 2E4 czhao@cs.toronto.edu Abstract Spam identification is crucial

### Classifiers & Classification

Classifiers & Classification Forsyth & Ponce Computer Vision A Modern Approach chapter 22 Pattern Classification Duda, Hart and Stork School of Computer Science & Statistics Trinity College Dublin Dublin

### How to Create and Manage your Junk Email Inbox Rule

How to Create and Manage your Junk Email Inbox Rule Overview Every email message received from outside of the University is scanned by a program called SpamAssassin before it is delivered to your Exchange

### MA 1125 Lecture 14 - Expected Values. Friday, February 28, 2014. Objectives: Introduce expected values.

MA 5 Lecture 4 - Expected Values Friday, February 2, 24. Objectives: Introduce expected values.. Means, Variances, and Standard Deviations of Probability Distributions Two classes ago, we computed the

### Secret 1: Never Buy or Use Mass Email Lists I was just reading a post on a well known marketing forum yesterday and someone said he

Legal Notice/Disclaimer There is no guarantee of success, traffic and/or that you will make money, either written or implied. The author /publisher specifically disclaims any personal liability, loss,

### Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

### A Logistic Regression Approach to Ad Click Prediction

A Logistic Regression Approach to Ad Click Prediction Gouthami Kondakindi kondakin@usc.edu Satakshi Rana satakshr@usc.edu Aswin Rajkumar aswinraj@usc.edu Sai Kaushik Ponnekanti ponnekan@usc.edu Vinit Parakh

### Email Spam Detection A Machine Learning Approach

Email Spam Detection A Machine Learning Approach Ge Song, Lauren Steimle ABSTRACT Machine learning is a branch of artificial intelligence concerned with the creation and study of systems that can learn

### CREATE TAX ADVANTAGED RETIREMENT INCOME YOU CAN T OUTLIVE. create tax advantaged retirement income you can t outlive

create tax advantaged retirement income you can t outlive 1 Table Of Contents Insurance Companies Don t Just Sell Insurance... 4 Life Insurance Investing... 5 Guarantees... 7 Tax Strategy How to Get Tax-Free

### How an Innovative Email Marketing Strategy Can Pave the Way to New Customers

How an Innovative Email Marketing Strategy Can Pave the Way to New Customers AN EMAIL MARKETING STRATEGY WHITE PAPER EMAIL MARKETING PROGRAMS According to the Direct Marketing Association, email marketing

### Review of Computer Engineering Research WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM. Geeta R.B.* Shobha R.B.

Review of Computer Engineering Research journal homepage: http://www.pakinsight.com/?ic=journal&journal=76 WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM Geeta R.B.* Department

### Lecture 6. Weight. Tension. Normal Force. Static Friction. Cutnell+Johnson: 4.8-4.12, second half of section 4.7

Lecture 6 Weight Tension Normal Force Static Friction Cutnell+Johnson: 4.8-4.12, second half of section 4.7 In this lecture, I m going to discuss four different kinds of forces: weight, tension, the normal

### Detecting Email Spam. MGS 8040, Data Mining. Audrey Gies Matt Labbe Tatiana Restrepo

Detecting Email Spam MGS 8040, Data Mining Audrey Gies Matt Labbe Tatiana Restrepo 5 December 2011 INTRODUCTION This report describes a model that may be used to improve likelihood of recognizing undesirable

### Towards running complex models on big data

Towards running complex models on big data Working with all the genomes in the world without changing the model (too much) Daniel Lawson Heilbronn Institute, University of Bristol 2013 1 / 17 Motivation

### The Social Accelerator Setup Guide

The Social Accelerator Setup Guide Welcome! Welcome to the Social Accelerator setup guide. This guide covers 2 ways to setup SA. Most likely, you will want to use the easy setup wizard. In that case, you

### Introduction to Learning & Decision Trees

Artificial Intelligence: Representation and Problem Solving 5-38 April 0, 2007 Introduction to Learning & Decision Trees Learning and Decision Trees to learning What is learning? - more than just memorizing

### GET THE WORKING CAPITAL YOU NEED

GET THE WORKING CAPITAL YOU NEED You ve sealed the deal, but now you need cash to start production. This is where your credibility is at stake, and EDC s solutions will help you show your customers that

### Small Business Owners: How You Can-and Must-Protect Your Business From The IRS If You Have Payroll Tax Problems!

E N O, B O U L A Y, M A R T I N & D O N A H U E, L L P SPECIAL SMALL BUSINESS TAX REPORT Inside this Special Report Small Business Owners: How You Can-and Must-Protect Your Business From The IRS If You

### 7 Secrets To Websites That Sell. By Alex Nelson

7 Secrets To Websites That Sell By Alex Nelson Website Secret #1 Create a Direct Response Website Did you know there are two different types of websites? It s true. There are branding websites and there

### The Gravity Model: Derivation and Calibration

The Gravity Model: Derivation and Calibration Philip A. Viton October 28, 2014 Philip A. Viton CRP/CE 5700 () Gravity Model October 28, 2014 1 / 66 Introduction We turn now to the Gravity Model of trip

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

### Chapter 6. The Payroll Profession

Chapter 6 The Payroll Profession The trick is in what one emphasizes. We either make ourselves miserable, or we make ourselves strong. The amount of work is the same Carlos Castaneda I am adamant in my

### Using Form Scripts in WEBPLUS

Using Form Scripts in WEBPLUS In WEBPLUS you have the built-in ability to create forms that can be sent to your email address via Serif Web Resources. This is a nice simple option that s easy to set up,

### How to make the Emails you Send from Outlook 2003 appear to Originate from different Email Addresses

How to make the Emails you Send from Outlook 2003 appear to Originate from different Email Addresses If you only use a single email address to send out all your business and personal emails then you're