Tracking change in word meaning

Transcription

1 Overview Intro DisSem Previous Case Visualisation Conclusion References Tracking change in word meaning A dynamic visualization of diachronic distributional semantics Kris Heylen, Thomas Wielfaert & Dirk Speelman KULeuven Quantitative Lexicology and Variational Linguistics

2 Purpose of the talk A lexicological study of how a set of near-synonymous adjectives have changed meaning through time, using a statistical, distributional approach for modelling lexical semantics in large corpora, using a dynamic visualization to assist in interpreting these statistical patterns, with the ultimate goal of creating an exploritative tool for lexical semantic analysis.

3 Overview 1. Background: Lexical variation 2. Distributional semantics 3. Previous Visualisations 4. Case Study: positive evaluative adjectives 5. Dynamic Visualisation of semantic change 6. Conclusion

5 Background: Lexical Variation LEXICOLOGY:

6 Background: Lexical Variation LEXICOLOGY

7 Background: Lexical Variation LEXICOLOGY: SEMASIOLOGICAL PERSPECTIVE

8 Background: Lexical Variation LEXICOLOGY: ONOMASIOLOGICAL PERSPECTIVE

9 Background: Lexical Variation LEXICOLOGY: FINER GRAINED ANALYSIS OF SEMANTIC FEA- TURES

10 Background: Lexical Variation LEXICOLOGY: FINER GRAINED ANALYSIS OF SEMANTIC FEA- TURES

11 Background: Lexical Variation LEXICOLOGY: LECTAL VARIATION

12 Background: Lexical Variation LEXICOLOGY: CHRONO-LECTAL (DIACHRONIC) VARIATION

13 Background: Lexical Variation LEXICOLOGY: QUANTITATIVE CORPUS ANALYSIS

15 Distributional models of lexical semantics Linguistic origin: Distributional Hypothesis You shall know a word by the company it keeps (Firth) a word s meaning can be induced from its co-occurring words long tradition of collocation studies in corpus linguistics Semantic Vector Spaces in Computational Linguistics standard technique in statistical NLP for the large-scale automatic modeling of (lexical) semantics aka Vector Spaces Models, Distributional Semantic Models, Word Spaces,... (cf Turney & Pantel 2010 for overview) generalised, large scale collocation analysis mainly used for automatic thesaurus extraction: words occurring in same contexts have similar meaning

16 Semantic Vector Spaces as models of word meaning Practical Which two words out of a set of three have the same meaning? ongeval, koffie, accident Occurrences in context from a corpus Op de Brusselse ring deed zich een ongeval met een vrachtwagen voor s Morgens drinkt hij een kop koffie met melk en suiker 2 bestuurders raakten gekwetst bij een ongeval met een vrachtwagen in de avondspits veroorzaakte een accident een kilometerslange file als vieruurtje serveert het hotel koffie en gebak voor de gasten de auto was betrokken in een accident met een dodelijke afloop Met winterbanden is het risico op een ongeval bij vriesweer veel kleiner

17 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval

18 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval vader raakte gekwetst bij een ongeval met een vrachtwagen op de

19 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval voor zeven uur veroorzaakte een ongeval een kilometerslange file richting Antwerpen

20 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval vrachtwagens waren betrokken bij het ongeval, dat meer dan tien slachtoffers

26 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval accident

31 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval accident koffie

35 auto slachtoffer vrachtwagen file gekwetst suiker melk kop ongeval accident koffie Which words are similar?

36 Distributional models of lexical semantics word by word similarity matrix ongeval accident koffie ongeval accident koffie

37 Distributional models of lexical semantics Geometrical metaphor: Semantic distance frequencies weighted by collocational strength (pmi) vectors projected in context feature space: Word Space cosine of angle between vectors as semantic similarity measure

38 Distributional semantics: lexical variation Bilectal Word Spaces Extend Word Space from one corpus to two corpora representative for different lects/varieties 2 context vectors for each word, one for each variety most words will have themselves as most similar word... BUT words with diverging semantic structure will not

40 Sagi, Kaufmann & Clark 2009

41 Rohrdantz, Hautli, Mayer etal. 2011

42 Hilpert 2011

44 Case study: : positive evaluative adjectives

45 Case study: positive evaluative adjectives brilliant cool delightful excellent fabulous fantastic good great impressive lovely magnificent marvelous perfect splendid superb terrific wonderful Table: positive evaluative adjectives

46 Case study Corpus Corpus of Historical American English (COHA, Davies 2012) Period from 1810 to 2009, 400M words, POS-tagged. Concept: Positive evaluative adjectives 1 vector per adjective, per decade ( ) modelled by window of 5 words left & right 5000 most frequent context words (minus top 100) PMI-weighting, cosine similarity

48 HighD to 2D Visualisation word-decade by context matrix is high dimensional first aim is NOT to find latent structure (as with LSA/LDA) but general picture of distributional semantic structuring faithful rendering of similarity matrix in 2D: Kruskal s non-metric Multidimensional Scaling interpret dimensions with context-labeled clusters Dynamic and interactive chart Motion Charts from Google Chart Tools panchronic view to interpret semantic space diachronic view to see meaning changes.

49 panchronic view for interpretation of semantic space Clusters with most typical contextwords of adjectives: cluster 2 (centre, light blue): positive evaluated things (colors, spectacle, performance) centre of the plot, expressing the core meaning of the adjectives cluster 8 (red, lower left): loud and frightening things (explosion, thunder, crash) periphery of the plot, expressing non-related meaning

50 diachronic motion chart to see meaning change Trajectory of terrific from 1860 to 2000, moving from the peripheral cluster of frightening things to the central cluster of positive evaluated things, indicative of its meaning change

52 Summary Conclusion and future work Lexicological perspective: Tool for exploring lexical semantics and variation in large amounts of corpus data Dynamic visualisation of evolving semantic structuring for a set of near-synonymous adjectives Desiderata integrate with latent dimension finding techniques (cf. Rohrdantz et al.) for easier interpretation of semantic space show individual occurrences of lexemes (tokens) to explore semasiological structure of adjectives in each decade show interpretative beacons in the dynamic plot other types of context features (e.g. dependency relations)

53 For more information:

54 References I Davies, Mark Corpus of Historical American English (COHA): ): 400+ million words, Heylen, Kris, Speelman, Dirk, & Geeraerts, Dirk Looking at word meaning. An interactive visualization of Semantic Vector Spaces for Dutch synsets. Pages of: Proceedings of the EACL-2012 joint workshop of LINGVIS & UNCLH: Visualization of Language Patters and Uncovering Language History from Multilingual Resources. Hilpert, Martin Dynamic visualizations of language change: Motion charts on the basis of bivariate and multivariate data from diachronic corpora. International Journal of Corpus Linguistics, 16(4),

55 References II Rohrdantz, Christian, Hautli, Annette, Mayer, Thomas, Butt, Miriam, Keim, Daniel A, & Plank, Frans Towards Tracking Semantic Change by Visual Analytics. Pages of: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Portland, Oregon, USA: Association for Computational Linguistics. Sagi, Eyal, Kaufmann, Stefan, & Clark, Brady Semantic Density Analysis: Comparing Word Meaning across Time and Phonetic Space. Pages of: Proceedings of the Workshop on Geometrical Models of Natural Language Semantics. Athens, Greece: Association for Computational Linguistics. Turney, Peter D., & Pantel, Patrick From Frequency to Meaning: Vector Space Models of Semantics. Journal of Artificial Intelligence Research, 37(1),