Department of Cognitive Sciences University of California, Irvine 1

Transcription

1 Mark Steyvers Department of Cognitive Sciences University of California, Irvine 1

2 Network structure of word associations Decentralized search in information networks Analogy between Google and word retrieval 2

3 n words = 5,000+

4 Categories 1,000 Word forms Word forms 29,000+

5 Word senses 99,000+ Word forms 122,000+

6 1. Short Path Lengths Word Association Roget WordNet n = number of nodes , ,000+ D = diameter L = average path length Local Clustering Neighbors of a node are each other s neighbors C = P( ) Power-Law degree distributions C=0 C=1

7 Exponential: Scale-free (Power law degree distributions) HUBS e.g., random graphs

8 P( k ) ROGET'S WORD ASSOCIATION WORDNET THESAURUS γ= γ= k 10-6 γ= k k

9 #meaning gs Slope in rank plot a=.466 Adamic (2000): γ=1+1/a Slope in distribution plot γ = 3.15 Word frequency rank

10 Small world properties put qualitative constraints on theories of semantic representations (Griffiths, Steyvers, Tenenbaum, 2007; Steyvers & Tenenbaum, 2005) 10

11 Why is network anatomy so important to characterize? Because structure always affects function. Strogatz (2001). Nature, 410, p. 268 Small worlds ld allow efficient i search processes 11

12 12

13 Random person in Nebraska Median of 6 steps ( 6 degrees of separation ) Target person in Boston

14 Key insight i from Milgram 1967: Short paths exist but can also be found using simple decentralized search algorithms No global knowledge of network is required, only notion of proximity to target Search time O( log n ) with small world structures Relevance for finding information on peer to peer networks without using a global index 14

15 Global memory models (e.g. Minerva, REM, etc) Proposed Model Target memory Memory trace Memory trace Memory trace Memory trace Cue Memory trace Memory trace Memory trace Memory trace Memory trace Cue Local connectivity No global index into memory Notion of proximity to target Decentralized search of target information

16 (Griffiths, Steyvers, Flirl, 2007; Psych Science) 16

17 World Wide Web Associative semantic networks Page Pet Cat Bone Dog Page Page Page Google PageRank uses network structure to rank documents Can we use PageRank to explain how humans retrieve words using semantic network structure? 17

18 18

19 How can Google know that we are interested in this page out of 377,000? 19

20 Measure for the relative importance of a webpage based on link structure Key idea: the relationship between importance and linking is recursive: a highly important webpage is a webpage that receives many links from other highly important webpages Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30,

21 Adjacency matrix L L ij = 1 if there is a link from document j to i Normalize matrix L to create a Markov chain PageRank values correspond to the principal eigenvector of L 21

22 A B C D Note: PageRank also assumes an additional process. At each time, there is a 0.15 probability of a teleport jump to any page in the network 22

23 P=.36 P=.34 P=.18 P=.12 Note: PageRank also assumes an additional process. At each time, there is a 0.15 probability of a teleport jump to any page in the network 23

24 PageRank a as the relative eat eimportance of a word od Does this explain word production in humans? Is it better than word frequency or other measures of availability? word word word word word od word 24

25 Cue participant with letter of alphabet. D Participants give first word that comes to mind: E.g. DOG, DAD, DOOR 50 participants, 21 letters of alphabet 25

26 A B C D P 26

27 Nelson et al. (1998) 5018 x 5018 matrix of associations Applied PageRank on unweighted semantic association matrix Pet Dog Cat Bone 27

28 28

29 Predictor All words PageRank 8.3% In degree 10.0% Word frequency (TASA) 19.0% 29

30 Predictor All words PageRank 8.3% In degree 10.0% Word frequency (TASA) 19.0% Weighted PageRank 7.1% Weighted in degree 8.2% 30

31 Predictor All words Nouns Only Concrete Nouns PageRank 8.3% 8.2% 13.3% In degree 10.0% 14.8% 17.5% Word frequency (TASA) 19.0% 22.5% 21.6% Weighted PageRank 7.1% 86% 8.6% 13.3% 33% Weighted in degree 8.2% 13.0% 16.7% 31

32 PageRank on word association might approximate psychological mechanisms Random walk on a semantic network Random mental surfing Bone Pet Cat Dog 32

33 Neighborhood h structure of words is useful lfor prediction Local structure e.g., number of incoming and outgoing connections (fan in, fan out) Global structure e.g. Google PageRank 33

34 Similar computational demands: Both retrieve the most relevant items from a large information ato repository in response se to external ete acues or queries. Useful analogies/ interdisciplinary i approaches 34