Topic Extrac,on from Online Reviews for Classifica,on and Recommenda,on (2013) R. Dong, M. Schaal, M. P. O Mahony, B. Smyth

Transcription

1 Topic Extrac,on from Online Reviews for Classifica,on and Recommenda,on (2013) R. Dong, M. Schaal, M. P. O Mahony, B. Smyth Lecture Algorithms to Analyze Big Data Speaker Hüseyin Dagaydin Heidelberg, 27 th January 2015

2 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

3 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

4 Intro [stat., 2013] 3

5 Mo,va,on [TripAdvisor, Amazon] 4

6 Effects Increasing number of user- generated reviews Overflow of informa,on Useless/unhelpful reviews 5

7 Two Naive Approaches 1. Equa,on by [Kim et al., 5, P. 424] 2. h(r R) = rating + (r) rating + (r)+ rating (r) = % 6

8 Naive Approach by Amazon Ranked by rating + (r) 1. s à % 2. S à 72 88% 3. s à 36 90% Why is 2 higher ranked than 3? 7

9 Recap & Goal Ranking depends on created,me stamp The older the review, the higher it will be ranked. To be listed in the ranking, ra,ng #r + or #r - has to be > 0 ü Goal: Ranking- System containing high- qualified helpful reviews 8

10 The Idea Input: Reviews Extract interes,ng topics out of the reviews Assign sen,ment labels to the topics Perform this opera,on automa,cally on raw textual data Output: Reviews linked with sen,ment tuples (R i, S j,t k,+ / / =) Review R, Sentence S, Topic T, posi,ve +, nega,ve -, neutral = 9

11 Example Digital Camera Green = posi,ve background Red = nega,ve background Topics are printed in bold 10

12 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

13 Topic Extrac,on & Sen,ment Analy. Our goal: Filter the topics from user- generated reviews Topics describe a product Assign sen,ment labels to the topics Posi,ve, neutral, nega,ve 12

14 Architecture (Reviews) 13

15 Part- of- Speech Tagging Assign each word to its corresponding word class/part- of- speech (POS) e.g The Mac Book Pro is a great laptop. DT NP V DT A N DT = Determiner N = Noun NP = Proper Noun A = Adjec,ve V = Verb 14

16 Architecture (Topic Extrac,on) 15

17 Topic Extrac,on - Basics Two types of topics Bi- grams Single- nouns 16

18 Topic Extrac,on Bi- gram Two consecu,ve words Two kinds: Adjec,ve Noun (AN), wide angle Noun Noun (NN), video mode excellent camera AN à!bi- gram à Candidate topics 17

19 Topic Extrac,on Single- nouns (1) All words consis,ng of non stop- word nouns Two steps for gewng the single- noun topics: 1. Crea,ng a candidate set by extrac,ng nouns - Problem: Oxen unqualified to be candidates - In July, I was with my family in NYC. - There are more words, such as 'me, day, vaca,on... - So, what to do with these words? 18

20 Topic Extrac,on Single- nouns (2) 2. Collect single- nouns sa,sfying the threshold condi,on - Sen,ment lexicon provided by [Hu & Liu, 2004] - For each C i single- noun - How frequently they appear nearby words from the list of sen,ment words? - Keep those with frequency > 70 % (=Threshold) 19

21 Result Set for Topic Extrac,on Two candidate sets of bi- gram and single- nouns Further filtering step: Keep those topics occuring in at least k reviews of totally n reviews à Topics (T 1,...,T m ) 20

22 Architecture (Sen,ment Analysis) 21

23 Sen,ment Analysis (1) The sen,ment lexicon is used Input: Topic T i, Sentence S j, Review R k For a given T i, determine all sen,ment words in a S j - If #sen,ment words == 0, then label T i as neutral 22

24 Sen,ment Analysis (2) If S j contains sen,ment words (w 1, w 2,...): - Iden,fy the word w min - w min = the word having the closest word- distance to T i - Define POS tags for w min, T i, and any words between those two words - Example:...this camera has a great noise reduc'on... T i = noise reduc'on, w min = great à POS sequence 23

25 Sen,ment Analysis (3) POS sequence has an Opinion Pa;ern (OP), e.g. JJ- Topic [Moghaddam and Ester, 2010] Frequency of different OPs - If frequency of OP > average number of occurences of all OPs à valid à posi,ve or nega,ve - otherwise, neutral 24

26 Architecture 25

27 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

28 Classifying Helpful Reviews Recap: What we have already done?! - Mining T i from R k - Associa,ng each R k with sen,ment tuples à (R i, S j, T k, +/- /=) Feature Set for classifica,on 27

29 Feature Set (1) Temporal Informa,on (AGE) Ra'ng Informa'on (RAT) à e.g. amazon.com Simple Sentence and Word Counts (SIZE) Topical Coverage (TOP) Sen>ment Informa>on (SENT) Readability Metrics (READ) The 50 most frequent topics of a par,cular product (CNT) Best Prac'ces 28

30 Topical Coverage (TOP) S S s 29

31 Sen,ment Informa,on (SENT) S 30

32 Feature Set (2) Expansion Feature sets contain features - i.e.: TOP à Breadth, Depth, TopicRank SENT à Density Review instances are represented by the feature set Example: Breadth of a R k might be 5. Is that a high or low value? We need further metrics... - Mean, Standard Devia,on, Normaliza,on 31

33 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

34 Datasets & Setup Review Set from amazon.com Ca Reviews for approximately different products 4 product categories - Digital Camera, GPS Devices, Laptops, Tablets Helpfulness Score à 0.7 Classifier: RF (Random Forest), JRip, Naive Bayes (NB) 33

35 Comparing the Features 34

36 Comparing the Classifiers 35

37 Outline (1) Introduc,on (2) Topic Extrac,on and Sen,ment Analysis Part- of- Speech Tagging Classifying Helpful Reviews (3) Evalua,on (4) Summary

38 Summary Extrac,ng topics from reviews Assigning sen,ment labels to the resul,ng topic set How to classify helpful reviews? à Feature Set Evalua,on Ra,ng Informa,on + Sen,ment Informa,on (=SENT- 2) > 0.7 AUC RF achieved the best score in the classifier comparison 37

39 THANK YOU

40

41 Resources Ruihai Dong et. al., 2013, Topic Extrac,on from Online Reviews for Classifica,on and Recommenda,on, Proceedings of the Twenty- Third interna,onal joint conference on Ar,ficial Intelligence. AAAI Press, University College Dublin Moghaddam Samaneh et.al., 2010, Opinion Digger: An Unsupervised Opinion Miner from Unstructured Product Reviews, Proceedings of the 19th ACM interna,onal conference on Informa,on and knowledge management. ACM NY S.- M. Kim et. al., 2006, Automa,cally assessing review helpfulness, In Proceedings of the Conference on Empirical Methods in Natural Language Processing, P , Sydney Anzahl der Personen in Deutschland, die das Internet zum Bestellen von Produkten und Dienstleistungen (Online- Shopping) nutzen, 2013, Sta,sta, h{p:// de.sta,sta.com/sta,s,k/daten/studie/183211/umfrage/online- shopping- - - internetnutzung