Social Media Analy.cs (SMA)

Transcription

1 Social Media Analy.cs (SMA) Emanuele Della Valle DEIB - Politecnico di Milano emanuele.dellavalle@polimi.it hap://emanueledellavalle.org

2 What's social media? haps:// Emanuele Della Valle hap://emanueledellavalle.org 2

3 shall I care? haps:// Emanuele Della Valle hap://emanueledellavalle.org 3

4 I'm s.ll unconvinced haps:// Emanuele Della Valle hap://emanueledellavalle.org 4

5 Personal Social Media Analy.cs Facebook Analy.cs haps:// /insights/ TwiAer Analy.cs haps://analy.cs.twiaer.com/ Any social media site has one, some.me it is pay per use, e.g., flickr More generally for any other Website under individual control, one can use google analy.cs hap:// 5

6 Facebook analy.cs 6 [source: haps:// guide- to- social- media/]

7 Facebook analy.cs cont. 7

8 Facebook analy.cs cont. 8

9 TwiAer Analy.cs 9 [source: hap://organizedassistant.com/5- recent- social- media- changes/ ]

10 Google Analy.cs 10 [source: haps://flic.kr/p/kjz4t]

11 What can you do as a company? The goal: reach à engage à acquire à mone.se How: Social media plan 11 [source: haps://

12 What can you do as a company? Unless you want to spend in manually integra.ng the personal social media tools seen before, you need a cross- plakorm tool Market hap://sysomos.com/ hap:// media- marke.ng/radian6 hap://hootsuite.com (the analy.cs part) hap:// (applicable to social media data) 12

13 sysomos.com 13 haps://youtu.be/trjo8w737i8

14 Radiant6 [haps://youtu.be/8i6exg3urg0] B 1/12/2014 hap://emanueledellavalle.org 14

15 Take home message Analysing social media you may be able to feel the pulse of the piece of reality genera.ng them and make sense of it 15

16 Is this new? [source: haps:// 16

17 No. So what's new? How pervasive the digital world is in the physical world There's a log way From web logs to App analy.cs of users sharing loca.ons! Big Data! We now have the enabling technology: 17

18 how does SMA relates with Big Data? Big Data technologies can scale this ability to the volume generated by a global contact centre, a na.on, the planet the velocity generated by a TV show, a sport event, IoT sensors the variety of en..es, languages or cultures the veracity problems caused by spelling errors, ambiguous words, irony/sarcasm Meaning of the colour: doable, possible, challenging 18

19 Key idea: from micropost to data Collect microposts Named En.ty Recogni.on En.ty Linking Sen.ment Opinion extrac.on Extract data 19

20 Facebook APIs Collect microposts haps://developers.facebook.com/ TwiAer APIs haps://dev.twiaer.com/docs/api Instagram APIs hap://instagram.com/developer/endpoints/ 20

21 Anatomy of a micropost sender Emanuele Della Valle Men.oned Happily lost in a boale of Heineken beer #heinekendesignweek at the Heineken Magazzini hap:// Link to a Web resource hap://emanueledellavalle.org 21 hashtag

22 Anatomy of a micropost There is already an amount of data here! Emanuele Della Bla bla bla bla bla bla bla bla bla bla bla bla #heinekendesignweek bla bla bla bla bla bla hap:// hap://emanueledellavalle.org 22

23 Emanuele Della Valle Anatomy of a micropost There is already an amount of data Bla bla bla bla bla bla bla bla bla bla bla bla #heinekendesignweek bla bla bla bla bla bla hap:// #heinekendesignweek hap://emanueledellavalle.org 23 type posts hashtag type link hap:// men.ons

24 Named En.ty Recogni.on (NER) Defini.on loca.ng and classifying atomic elements [...] into predefined categories such as names, persons, organiza.ons, loca.ons, expressions of.me, quan..es, monetary values, etc. A.k.a. C.J.Rijsbergen, Informa.on Retrieval (1979) En.ty Iden.fica.on En.ty Extrac.on 24

25 Named En.ty Recogni.on, example Input: Armstrong landed the Eagle on the Moon. Possible outputs: Armstrong Eagle Moon Person: animal. Company Band (misspelled). Character 25 Vehicle

26 Named En.ty Recogni.on, example Input: Armstrong landed the Eagle on the Moon. Desired Output: Armstrong: person Eagle: vehicle Moon: celes.al body Note: a very hard task :- ( Early solu.ons were based on manually wriaen grammars Modern solu.ons are base on Machine Learning Advance solu.ons perform NER together with En.ty linking 26

27 Defini.on En.ty Linking (EL) It is the task of determining the iden.ty of en..es men.oned in text. It is dis.nct from named en.ty recogni.on (NER) in that it iden.fies not the occurrence of names (and a limited classifica.on of those), but their reference. 27 haps://en.wikipedia.org/wiki/en.ty_linking

28 En.ty Linking - process Determine all possible en.ty links candidates Select the best en.ty link 28

29 En.ty Linking - process Determine all possible En.ty Mapping Candidates linguis.c analysis, i.e., part- of- speech (POS) tagging Normaliza.on encoding and spelling special (language dependent) characters language dependent spellings abbrevia.ons, acronyms type dependent spellings alterna.ve names and synonyms fuzzy string mapping 29

30 En.ty Linking - process Select the best en..es links from all possible candidate ones: Popularity/sta=s=c based strategies based on a reference text corpus Linguis=c/Seman=c based strategies based on a knowledge base/graph At high level Make an assump.on Run strategies in parallel 30 Decide using logical/ probabilis.c rules

31 En.ty Linking - example Determine candidates links Input: Armstrong landed the Eagle on the Moon. B 1/12/2014 [source: hap:// hpi- semweb06part ] hap://emanueledellavalle.org 31

32 En.ty Linking - example Select Links Input: Armstrong landed the Eagle on the Moon. B 1/12/2014 [source: hap:// hpi- semweb06part ] hap://emanueledellavalle.org 32

33 En.ty Linking - example Input: Armstrong landed the Eagle on the Eagle. Output: Armstrong hap://dbpedia.org/resource/neil_armstrong Eagle hap://dbpedia.org/resource/apollo_lunar_module Moon hap://dbpedia.org/resource/moon 33

34 Applying NER From micropost to data drink Emanuele Della Bla bla bla bla bla bla bla bla Heineken beer #heinekendesignweek bla the bla bla Magazzini hap:// hap://emanueledellavalle.org 34 company Place event

35 Company Drink From micropost to data Adding the data to those we got before Emanuele Della Valle Place Event type type type type User "Heineken" type "beer" "The Magazzini" "#heinekendesignweek" topic topic topic posts micropost42 link hap://emanueledellavalle.org 35 men.ons hap://

36 From micropost to Bla bla bla bla bla bla bla bla Heineken beer #heinekendesignweek hap://wordnet Emanuele Della Valle linking en..es bla the bla bla Magazzini hap:// hap://emanueledellavalle.org 36 hap://dbpedia.org/ hap://fuorisalone hap://fuorisalone

37 From micropost to data Adding the data (which are also linked) to those we got before User Company type Heineken topic type type Drink Emanuele Della Valle Place Event type type type bear The Magazzini Heineken design week topic topic topic posts micropost42 link men.ons hap://

38 From micropost to data Adding the data (which are also linked) to those we got before Heineken User type follows produces topic type organizes Emanuele Della Valle offers Located at Heineken design week beer The Magazzini topic topic topic posts micropost42 link men.ons hap:// offers

39 Adding sen.ment and opinion Machine learning Given a large mount of training data Classify unseen data Dic.onary approach Sen.WordNet hap://sen.wordnet.is..cnr.it/ WordNet- Affect hap://wndomains.xk.eu/wnaffect.html Advance approaches (combines the previous two) Sta.s.cal Approach Seman.c Approach 39

40 linking en..es From micropost to data Posi.ve opinion Emanuele Della Happily bla bla bla bla bla bla Heineken beer #heinekendesignweek bla the bla bla Magazzini hap:// hap://emanueledellavalle.org 40

41 From micropost to data Adding sen.ment / opinion User produces Heineken Posi=ve topic type type follows organizes Emanuele Della Valle offers Located at Heineken design week beer The Magazzini topic topic topic posts micropost42 link men.ons sen=ment hap:// 0.9 offers

42 Adding sen.ment and opinion Discussion about precision Hard also for humans as well! Sarcasm Irony Be aware of contradic.ons and value them The food and the service are good, but the service sucks 42

43 Tools hap:// hap://viewer.opencalais.com/ hap:// hap://dandelion.eu/ (Italian is supported!) 43

44 Key idea: from micropost to data Collect microposts Named En.ty Recogni.on En.ty Linking Sen.ment Opinion extrac.on Extract data 44

45 Acknowledges OpenHPI Named En.ty Recogni.on, by Harald Sack hap:// hpi- semweb06part

46 Thank you! Any Ques.on? Emanuele Della Valle DEIB - Politecnico di Milano emanuele.dellavalle@polimi.it hap://emanueledellavalle.org