Multi-way classification

Size: px

Start display at page:

Download "Multi-way classification"

Leon O’Brien’
8 years ago
Views:

1 CS 75 Mache Learg Lecture Mult-a classfcato Mlos Hauskrecht 59 Seott Suare CS 75 Mache Learg Admstratve aoucemets Homeork 6 due o Wedesda Pla for the ucomg moth: Homeork 7 due o Wedesda //4 Mdterm: /7/4 Proect roosals: /5/4 CS 75 Mache Learg

edu 59 Seott Suare CS 75 Mache Learg Admstratve aoucemets

2 Mult-a classfcato Bar classfcato Y {} Mult-a classfcato K classes Y { K K } Goal: lear to classf correctl K classes Or lear f : X { K K } Errors: Zero-oe msclassfcato error for a eamle: f Error f Mea msclassfcato error for a dataset: Error CS 75 Mache Learg Mult-a classfcato Aroaches: Dscrmatve aroach Parametrc dscrmat fuctos Lears dscrmat fuctos drectl A logstc regresso model. Geeratve model aroach Geeratve model of the dstrbuto Lears the arameters of the model through dest estmato techues Dscrmat fuctos defed o the to of the model Idrect learg of a classfer CS 75 Mache Learg

Dscrmatve aroach Parametrc dscrmat fuctos Lears dscrmat fuctos drectl A logstc regresso model.

3 Geeratve model aroach Idrect:. Rereset ad lear the dstrbuto. Defe ad use robablstc dscrmat fuctos g log Model Class-codtoal dstrbutos destes k class-codtoal dstrbutos K Prors o classes - robablt of class K CS 75 Mache Learg Mult-a classfcato. Eamle CS 75 Mache Learg

destes k class-codtoal dstrbutos K Prors o classes - robablt of class K

4 Mult-a classfcato CS 75 Mache Learg Dscrmatve aroach. Parameterc model of dscrmat fuctos Lears the dscrmat fuctos drectl Ho to lear to classf multle classes sa? Aroach : A bar logstc regresso o ever class versus the rest vs. or d vs. or vs. or CS 75 Mache Learg

drectl Ho to lear to classf multle classes sa?

5 Mult-a classfcato. Eamle CS 75 Mache Learg Mult-a classfcato. Aroach..5 vs {}.5 vs {}.5 vs {} CS 75 Mache Learg

6 Mult-a classfcato. Aroach..5 Ambguous.5 rego.5 vs {} vs {} Rego of obod vs {} CS 75 Mache Learg Mult-a classfcato. Aroach..5 vs {} vs {}.5.5 vs {} CS 75 Mache Learg

Aroach..5 vs {} vs {}.5.5 vs {} -.5 - - -.5 - -.5.5.5 CS 75 Mache Learg

7 Dscrmatve aroach. Ho to lear to classf multle classes sa? Aroach : A bar logstc regresso o all ars d vs. vs. vs. CS 75 Mache Learg Mult-a classfcato. Eamle CS 75 Mache Learg

8 Mult-a classfcato. Aroach.5 vs.5 vs vs CS 75 Mache Learg Mult-a classfcato. Aroach.5 vs.5.5 vs Ambguous rego vs CS 75 Mache Learg

Aroach.5 vs.5.5 vs Ambguous rego -.5 - - -.

9 Mult-a classfcato. Aroach.5 vs.5 vs vs CS 75 Mache Learg Mult-a classfcato th softma A soluto to the roblem of havg a ambguous rego d e e CS 75 Mache Learg softma µ µ µ µ µ z z z

10 Mult-a classfcato th softma CS 75 Mache Learg Learg of the softma model Learg of arameters : statstcal ve Softma etork µ P µ k P k Mult-a Co toss Assume oututs are trasformed as follos {.. } k CS 75 Mache Learg

11 CS 75 Mache Learg Learg of the softma model Learg of the arameters : statstcal ve Lkelhood of oututs We at arameters that mamze the lkelhood Log-lkelhood trck Otmze log-lkelhood of oututs stead: Obectve to otmze D l.... log log log k D J µ k k.... log log µ µ.. X Y D L CS 75 Mache Learg Learg of the softma model Error to otmze: Gradet he same ver eas gradet udate as used for the bar logstc regresso But o e have to udate eghts of k etorks k D J µ log k D J µ + µ α

... log log log k D J µ k k.... log log µ µ.

12 CS 75 Mache Learg Mult-a classfcato Whe s the softma the rght model? Assume: P µ k P µ k Softma etork e a A h - locato arameter for class-codtoal - scalg arameter the same for all classes CS 75 Mache Learg Mult-a classfcato Class codtoal: Class osteror: + + b b a A h a A h e e e e a l a A b + e a A h

class-codtoal - scalg arameter the same for all classes CS 75 Mache Learg

13 Mult-a classfcato Softma model s a accurate model he class-codtoal destes are rereseted th destes from the eoetal faml th the same scalg arameter µ P Softma etork µ k P k b e + c a - locato arameter for class-codtoal - scalg arameter the same for all classes CS 75 Mache Learg CS 75 Mache Learg Lecture b Nearest-eghbor classfers Mlos Hauskrecht mlos@cs.tt.edu 59 Seott Suare CS 75 Mache Learg

arameter for class-codtoal - scalg arameter the same for all classes CS 75 Mache Learg CS 75 Mache

14 Classfcato roblem Class Class e ot What class label ould ou assg to the e data ot? CS 75 Mache Learg Nearest eghbor classfcato Class Class e ot Class sce there are more Class ots ts eghborhood CS 75 Mache Learg

15 Nearest eghbor classfcato Classfcato: Memor based use all eamles the data drectl As oosed to a arametrc models hch the effect of data s catured b arameters ad ther values Ramfcatos: No learg otmzato of arameters s ecessar All ork s doe at the tme of redcto Problems: Who are the eghbors? We eed a metrc to defe the eghborhood. CS 75 Mache Learg Nearest eghbor classfcato Eamle of a smle metrc: Eucldea d D ' ' Nearest eghbor classfcato: K-earest eghbors: use k eamles closest to Nearest eghbor: use a sgle eamle closest to Decso: A smle maort vote o k eamles closest to A eghted maort vote o k eamles A eght defes a mortace of a ot Imortace terms of a dstace CS 75 Mache Learg

CS 75 Mache Learg Nearest eghbor classfcato Eamle of a smle metrc: Eucldea d D ' ' Nearest eghbor classfcato: K-earest eghbors: use k eamles closest to Nearest eghbor:

16 Nearest eghbor classfcato A eghted maort vote o k eamles A eght defes a mortace of a ot Imortace terms of a dstace A eamle: A set of k eamles closest to the target th the dstace D A dataot comes th the eght Decso: add eghts for the same target label choose the er k u D D u CS 75 Mache Learg

closest to the target th the dstace D A dataot comes th the eght Decso:

CIS603 - Artificial Intelligence. Logistic regression. (some material adopted from notes by M. Hauskrecht) CIS603 - AI. Supervised learning

CIS603 - Artificial Intelligence. Logistic regression. (some material adopted from notes by M. Hauskrecht) CIS603 - AI. Supervised learning CIS63 - Artfcal Itellgece Logstc regresso Vasleos Megalookoomou some materal adopted from otes b M. Hauskrecht Supervsed learg Data: D { d d.. d} a set of eamples d < > s put vector ad s desred output