Department of Cognitive Sciences University of California, Irvine 1

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Department of Cognitive Sciences University of California, Irvine 1"

Transcription

1 Mark Steyvers Department of Cognitive Sciences University of California, Irvine 1

2 Network structure of word associations Decentralized search in information networks Analogy between Google and word retrieval 2

3 n words = 5,000+

4 Categories 1,000 Word forms Word forms 29,000+

5 Word senses 99,000+ Word forms 122,000+

6 1. Short Path Lengths Word Association Roget WordNet n = number of nodes , ,000+ D = diameter L = average path length Local Clustering Neighbors of a node are each other s neighbors C = P( ) Power-Law degree distributions C=0 C=1

7 Exponential: Scale-free (Power law degree distributions) HUBS e.g., random graphs

8 P( k ) ROGET'S WORD ASSOCIATION WORDNET THESAURUS γ= γ= k 10-6 γ= k k

9 #meaning gs Slope in rank plot a=.466 Adamic (2000): γ=1+1/a Slope in distribution plot γ = 3.15 Word frequency rank

10 Small world properties put qualitative constraints on theories of semantic representations (Griffiths, Steyvers, Tenenbaum, 2007; Steyvers & Tenenbaum, 2005) 10

11 Why is network anatomy so important to characterize? Because structure always affects function. Strogatz (2001). Nature, 410, p. 268 Small worlds ld allow efficient i search processes 11

12 12

13 Random person in Nebraska Median of 6 steps ( 6 degrees of separation ) Target person in Boston

14 Key insight i from Milgram 1967: Short paths exist but can also be found using simple decentralized search algorithms No global knowledge of network is required, only notion of proximity to target Search time O( log n ) with small world structures Relevance for finding information on peer to peer networks without using a global index 14

15 Global memory models (e.g. Minerva, REM, etc) Proposed Model Target memory Memory trace Memory trace Memory trace Memory trace Cue Memory trace Memory trace Memory trace Memory trace Memory trace Cue Local connectivity No global index into memory Notion of proximity to target Decentralized search of target information

16 (Griffiths, Steyvers, Flirl, 2007; Psych Science) 16

17 World Wide Web Associative semantic networks Page Pet Cat Bone Dog Page Page Page Google PageRank uses network structure to rank documents Can we use PageRank to explain how humans retrieve words using semantic network structure? 17

18 18

19 How can Google know that we are interested in this page out of 377,000? 19

20 Measure for the relative importance of a webpage based on link structure Key idea: the relationship between importance and linking is recursive: a highly important webpage is a webpage that receives many links from other highly important webpages Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30,

21 Adjacency matrix L L ij = 1 if there is a link from document j to i Normalize matrix L to create a Markov chain PageRank values correspond to the principal eigenvector of L 21

22 A B C D Note: PageRank also assumes an additional process. At each time, there is a 0.15 probability of a teleport jump to any page in the network 22

23 P=.36 P=.34 P=.18 P=.12 Note: PageRank also assumes an additional process. At each time, there is a 0.15 probability of a teleport jump to any page in the network 23

24 PageRank a as the relative eat eimportance of a word od Does this explain word production in humans? Is it better than word frequency or other measures of availability? word word word word word od word 24

25 Cue participant with letter of alphabet. D Participants give first word that comes to mind: E.g. DOG, DAD, DOOR 50 participants, 21 letters of alphabet 25

26 A B C D P 26

27 Nelson et al. (1998) 5018 x 5018 matrix of associations Applied PageRank on unweighted semantic association matrix Pet Dog Cat Bone 27

28 28

29 Predictor All words PageRank 8.3% In degree 10.0% Word frequency (TASA) 19.0% 29

30 Predictor All words PageRank 8.3% In degree 10.0% Word frequency (TASA) 19.0% Weighted PageRank 7.1% Weighted in degree 8.2% 30

31 Predictor All words Nouns Only Concrete Nouns PageRank 8.3% 8.2% 13.3% In degree 10.0% 14.8% 17.5% Word frequency (TASA) 19.0% 22.5% 21.6% Weighted PageRank 7.1% 86% 8.6% 13.3% 33% Weighted in degree 8.2% 13.0% 16.7% 31

32 PageRank on word association might approximate psychological mechanisms Random walk on a semantic network Random mental surfing Bone Pet Cat Dog 32

33 Neighborhood h structure of words is useful lfor prediction Local structure e.g., number of incoming and outgoing connections (fan in, fan out) Global structure e.g. Google PageRank 33

34 Similar computational demands: Both retrieve the most relevant items from a large information ato repository in response se to external ete acues or queries. Useful analogies/ interdisciplinary i approaches 34

The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth

The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth Cognitive Science 29 (2005) 41 78 Copyright 2005 Cognitive Science Society, Inc. All rights reserved. The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth

More information

Google and the Mind. Predicting Fluency With PageRank. Research Article. Thomas L. Griffiths, 1 Mark Steyvers, 2 and Alana Firl 3

Google and the Mind. Predicting Fluency With PageRank. Research Article. Thomas L. Griffiths, 1 Mark Steyvers, 2 and Alana Firl 3 PSYCHOLOGICAL SCIENCE Research Article Google and the Mind Predicting Fluency With PageRank Thomas L. Griffiths, 1 Mark Steyvers, 2 and Alana Firl 3 1 Department of Psychology, University of California,

More information

Asking Hard Graph Questions. Paul Burkhardt. February 3, 2014

Asking Hard Graph Questions. Paul Burkhardt. February 3, 2014 Beyond Watson: Predictive Analytics and Big Data U.S. National Security Agency Research Directorate - R6 Technical Report February 3, 2014 300 years before Watson there was Euler! The first (Jeopardy!)

More information

Search engines: ranking algorithms

Search engines: ranking algorithms Search engines: ranking algorithms Gianna M. Del Corso Dipartimento di Informatica, Università di Pisa, Italy ESP, 25 Marzo 2015 1 Statistics 2 Search Engines Ranking Algorithms HITS Web Analytics Estimated

More information

In the following we will only consider undirected networks.

In the following we will only consider undirected networks. Roles in Networks Roles in Networks Motivation for work: Let topology define network roles. Work by Kleinberg on directed graphs, used topology to define two types of roles: authorities and hubs. (Each

More information

Sociology and CS. Small World. Sociology Problems. Degree of Separation. Milgram s Experiment. How close are people connected? (Problem Understanding)

Sociology and CS. Small World. Sociology Problems. Degree of Separation. Milgram s Experiment. How close are people connected? (Problem Understanding) Sociology Problems Sociology and CS Problem 1 How close are people connected? Small World Philip Chan Problem 2 Connector How close are people connected? (Problem Understanding) Small World Are people

More information

An Alternative Web Search Strategy? Abstract

An Alternative Web Search Strategy? Abstract An Alternative Web Search Strategy? V.-H. Winterer, Rechenzentrum Universität Freiburg (Dated: November 2007) Abstract We propose an alternative Web search strategy taking advantage of the knowledge on

More information

Practical Graph Mining with R. 5. Link Analysis

Practical Graph Mining with R. 5. Link Analysis Practical Graph Mining with R 5. Link Analysis Outline Link Analysis Concepts Metrics for Analyzing Networks PageRank HITS Link Prediction 2 Link Analysis Concepts Link A relationship between two entities

More information

Tutorial, IEEE SERVICE 2014 Anchorage, Alaska

Tutorial, IEEE SERVICE 2014 Anchorage, Alaska Tutorial, IEEE SERVICE 2014 Anchorage, Alaska Big Data Science: Fundamental, Techniques, and Challenges (Data Mining on Big Data) 2014. 6. 27. By Neil Y. Yen Presented by Incheon Paik University of Aizu

More information

Ranking on Data Manifolds

Ranking on Data Manifolds Ranking on Data Manifolds Dengyong Zhou, Jason Weston, Arthur Gretton, Olivier Bousquet, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany {firstname.secondname

More information

DECENTRALIZED SCALE-FREE NETWORK CONSTRUCTION AND LOAD BALANCING IN MASSIVE MULTIUSER VIRTUAL ENVIRONMENTS

DECENTRALIZED SCALE-FREE NETWORK CONSTRUCTION AND LOAD BALANCING IN MASSIVE MULTIUSER VIRTUAL ENVIRONMENTS DECENTRALIZED SCALE-FREE NETWORK CONSTRUCTION AND LOAD BALANCING IN MASSIVE MULTIUSER VIRTUAL ENVIRONMENTS Markus Esch, Eric Tobias - University of Luxembourg MOTIVATION HyperVerse project Massive Multiuser

More information

A probabilistic approach to semantic representation

A probabilistic approach to semantic representation A probabilistic approach to semantic representation Thomas L. Griffiths & Mark Steyvers {gruffydd,msteyver}@psych.stanford.edu Department of Psychology Stanford University Stanford, CA 94305-2130 USA Abstract

More information

Scientific Collaboration Networks in China s System Engineering Subject

Scientific Collaboration Networks in China s System Engineering Subject , pp.31-40 http://dx.doi.org/10.14257/ijunesst.2013.6.6.04 Scientific Collaboration Networks in China s System Engineering Subject Sen Wu 1, Jiaye Wang 1,*, Xiaodong Feng 1 and Dan Lu 1 1 Dongling School

More information

Search engine ranking

Search engine ranking Proceedings of the 7 th International Conference on Applied Informatics Eger, Hungary, January 28 31, 2007. Vol. 2. pp. 417 422. Search engine ranking Mária Princz Faculty of Technical Engineering, University

More information

Network Analysis and Visualization of Staphylococcus aureus. by Russ Gibson

Network Analysis and Visualization of Staphylococcus aureus. by Russ Gibson Network Analysis and Visualization of Staphylococcus aureus by Russ Gibson Network analysis Based on graph theory Probabilistic models (random graphs) developed by Erdős and Rényi in 1959 Theory and tools

More information

A Graph Search Heuristic for Shortest Distance Paths

A Graph Search Heuristic for Shortest Distance Paths A Graph Search Heuristic for Shortest Distance Paths Edmond Chow Center for Applied Scientific Computing Lawrence Livermore National Laboratory Box 808, L-560, Livermore, CA 94551, USA echow@llnl.gov Abstract

More information

Introduction to Networks and Business Intelligence

Introduction to Networks and Business Intelligence Introduction to Networks and Business Intelligence Prof. Dr. Daning Hu Department of Informatics University of Zurich Sep 17th, 2015 Outline Network Science A Random History Network Analysis Network Topological

More information

Part 1: Link Analysis & Page Rank

Part 1: Link Analysis & Page Rank Chapter 8: Graph Data Part 1: Link Analysis & Page Rank Based on Leskovec, Rajaraman, Ullman 214: Mining of Massive Datasets 1 Exam on the 5th of February, 216, 14. to 16. If you wish to attend, please

More information

Social Media Mining. Network Measures

Social Media Mining. Network Measures Klout Measures and Metrics 22 Why Do We Need Measures? Who are the central figures (influential individuals) in the network? What interaction patterns are common in friends? Who are the like-minded users

More information

International Journal of Engineering Research-Online A Peer Reviewed International Journal Articles are freely available online:http://www.ijoer.

International Journal of Engineering Research-Online A Peer Reviewed International Journal Articles are freely available online:http://www.ijoer. RESEARCH ARTICLE SURVEY ON PAGERANK ALGORITHMS USING WEB-LINK STRUCTURE SOWMYA.M 1, V.S.SREELAXMI 2, MUNESHWARA M.S 3, ANIL G.N 4 Department of CSE, BMS Institute of Technology, Avalahalli, Yelahanka,

More information

Enhancing the Ranking of a Web Page in the Ocean of Data

Enhancing the Ranking of a Web Page in the Ocean of Data Database Systems Journal vol. IV, no. 3/2013 3 Enhancing the Ranking of a Web Page in the Ocean of Data Hitesh KUMAR SHARMA University of Petroleum and Energy Studies, India hkshitesh@gmail.com In today

More information

Social Networks and Social Media

Social Networks and Social Media Social Networks and Social Media Social Media: Many-to-Many Social Networking Content Sharing Social Media Blogs Microblogging Wiki Forum 2 Characteristics of Social Media Consumers become Producers Rich

More information

Overlay and P2P Networks. On power law networks. Prof. Sasu Tarkoma

Overlay and P2P Networks. On power law networks. Prof. Sasu Tarkoma Overlay and P2P Networks On power law networks Prof. Sasu Tarkoma 28.1.2013 Contents Zipf s law and power law distributions Robustness of networks Scale free and small worlds Search in small worlds Freenet

More information

Ranking Biomedical Passages for Relevance and Diversity

Ranking Biomedical Passages for Relevance and Diversity Ranking Biomedical Passages for Relevance and Diversity University of Wisconsin-Madison at TREC Genomics 2006 Andrew B. Goldberg,, David Andrzejewski, Jurgen Van Gael, Burr Settles, Xiaojin Zhu, Mark Craven

More information

RANKING AND VISUALIZING THE KEYWORDS IN MISSION STATEMENTS OF SLOVENIAN COMPANIES

RANKING AND VISUALIZING THE KEYWORDS IN MISSION STATEMENTS OF SLOVENIAN COMPANIES RANKING AND VISUALIZING THE KEYWORDS IN MISSION STATEMENTS OF SLOVENIAN COMPANIES Kristijan Breznik International School for Social and Business Studies, Slovenia kristijan.breznik@mfdps.si Abstract: Mission

More information

Graph Processing and Social Networks

Graph Processing and Social Networks Graph Processing and Social Networks Presented by Shu Jiayu, Yang Ji Department of Computer Science and Engineering The Hong Kong University of Science and Technology 2015/4/20 1 Outline Background Graph

More information

1 Contact:

1 Contact: 1 Contact: fowler@panam.edu Visualizing the Web as Hubs and Authorities Richard H. Fowler 1 and Tarkan Karadayi Technical Report CS-02-27 Department of Computer Science University of Texas Pan American

More information

A comparative study of social network analysis tools

A comparative study of social network analysis tools Membre de Membre de A comparative study of social network analysis tools David Combe, Christine Largeron, Előd Egyed-Zsigmond and Mathias Géry International Workshop on Web Intelligence and Virtual Enterprises

More information

The world s largest matrix computation. (This chapter is out of date and needs a major overhaul.)

The world s largest matrix computation. (This chapter is out of date and needs a major overhaul.) Chapter 7 Google PageRank The world s largest matrix computation. (This chapter is out of date and needs a major overhaul.) One of the reasons why Google TM is such an effective search engine is the PageRank

More information

Graph Theory and Complex Networks: An Introduction. Chapter 08: Computer networks

Graph Theory and Complex Networks: An Introduction. Chapter 08: Computer networks Graph Theory and Complex Networks: An Introduction Maarten van Steen VU Amsterdam, Dept. Computer Science Room R4.20, steen@cs.vu.nl Chapter 08: Computer networks Version: March 3, 2011 2 / 53 Contents

More information

Social Network Discovery based on Sensitivity Analysis

Social Network Discovery based on Sensitivity Analysis Social Network Discovery based on Sensitivity Analysis Tarik Crnovrsanin, Carlos D. Correa and Kwan-Liu Ma Department of Computer Science University of California, Davis tecrnovrsanin@ucdavis.edu, {correac,ma}@cs.ucdavis.edu

More information

1 o Semestre 2007/2008

1 o Semestre 2007/2008 Departamento de Engenharia Informática Instituto Superior Técnico 1 o Semestre 2007/2008 Outline 1 2 3 4 5 Outline 1 2 3 4 5 Exploiting Text How is text exploited? Two main directions Extraction Extraction

More information

Social Network Mining

Social Network Mining Social Network Mining Data Mining November 11, 2013 Frank Takes (ftakes@liacs.nl) LIACS, Universiteit Leiden Overview Social Network Analysis Graph Mining Online Social Networks Friendship Graph Semantics

More information

The Network Structure of Hard Combinatorial Landscapes

The Network Structure of Hard Combinatorial Landscapes The Network Structure of Hard Combinatorial Landscapes Marco Tomassini 1, Sebastien Verel 2, Gabriela Ochoa 3 1 University of Lausanne, Lausanne, Switzerland 2 University of Nice Sophia-Antipolis, France

More information

Gephi Network Statistics

Gephi Network Statistics Gephi Network Statistics Google Summer of Code 2009 Project Proposal Patrick J. McSweeney pjmcswee@syr.edu 1 Introduction My name is Patrick J. McSweeney and I am a fourth year PhD candidate in computer

More information

Perron vector Optimization applied to search engines

Perron vector Optimization applied to search engines Perron vector Optimization applied to search engines Olivier Fercoq INRIA Saclay and CMAP Ecole Polytechnique May 18, 2011 Web page ranking The core of search engines Semantic rankings (keywords) Hyperlink

More information

1 Six Degrees of Separation

1 Six Degrees of Separation Networks: Spring 2007 The Small-World Phenomenon David Easley and Jon Kleinberg April 23, 2007 1 Six Degrees of Separation The small-world phenomenon the principle that we are all linked by short chains

More information

Affinity Prediction in Online Social Networks

Affinity Prediction in Online Social Networks Affinity Prediction in Online Social Networks Matias Estrada and Marcelo Mendoza Skout Inc., Chile Universidad Técnica Federico Santa María, Chile Abstract Link prediction is the problem of inferring whether

More information

A Performance Evaluation of Open Source Graph Databases. Robert McColl David Ediger Jason Poovey Dan Campbell David A. Bader

A Performance Evaluation of Open Source Graph Databases. Robert McColl David Ediger Jason Poovey Dan Campbell David A. Bader A Performance Evaluation of Open Source Graph Databases Robert McColl David Ediger Jason Poovey Dan Campbell David A. Bader Overview Motivation Options Evaluation Results Lessons Learned Moving Forward

More information

6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III

6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III 6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these

More information

The PageRank Citation Ranking: Bring Order to the Web

The PageRank Citation Ranking: Bring Order to the Web The PageRank Citation Ranking: Bring Order to the Web presented by: Xiaoxi Pang 25.Nov 2010 1 / 20 Outline Introduction A ranking for every page on the Web Implementation Convergence Properties Personalized

More information

Social Network Analysis

Social Network Analysis Social Network Analysis Challenges in Computer Science April 1, 2014 Frank Takes (ftakes@liacs.nl) LIACS, Leiden University Overview Context Social Network Analysis Online Social Networks Friendship Graph

More information

Walk-Based Centrality and Communicability Measures for Network Analysis

Walk-Based Centrality and Communicability Measures for Network Analysis Walk-Based Centrality and Communicability Measures for Network Analysis Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, Georgia, USA Workshop on Innovative Clustering

More information

Collective Behavior Prediction in Social Media. Lei Tang Data Mining & Machine Learning Group Arizona State University

Collective Behavior Prediction in Social Media. Lei Tang Data Mining & Machine Learning Group Arizona State University Collective Behavior Prediction in Social Media Lei Tang Data Mining & Machine Learning Group Arizona State University Social Media Landscape Social Network Content Sharing Social Media Blogs Wiki Forum

More information

The Mathematics of Internet Search Engines

The Mathematics of Internet Search Engines The Mathematics of Internet Search Engines David Marshall Department of Mathematics Monmouth University April 4, 2007 Introduction Search Engines, Then and Now Then... Now... Pagerank Outline Introduction

More information

Extracting Information from Social Networks

Extracting Information from Social Networks Extracting Information from Social Networks Aggregating site information to get trends 1 Not limited to social networks Examples Google search logs: flu outbreaks We Feel Fine Bullying 2 Bullying Xu, Jun,

More information

Web Graph Analyzer Tool

Web Graph Analyzer Tool Web Graph Analyzer Tool Konstantin Avrachenkov INRIA Sophia Antipolis 2004, route des Lucioles, B.P.93 06902, France Email: K.Avrachenkov@sophia.inria.fr Danil Nemirovsky St.Petersburg State University

More information

Six Degrees of Separation in Online Society

Six Degrees of Separation in Online Society Six Degrees of Separation in Online Society Lei Zhang * Tsinghua-Southampton Joint Lab on Web Science Graduate School in Shenzhen, Tsinghua University Shenzhen, Guangdong Province, P.R.China zhanglei@sz.tsinghua.edu.cn

More information

A discussion of Statistical Mechanics of Complex Networks P. Part I

A discussion of Statistical Mechanics of Complex Networks P. Part I A discussion of Statistical Mechanics of Complex Networks Part I Review of Modern Physics, Vol. 74, 2002 Small Word Networks Clustering Coefficient Scale-Free Networks Erdös-Rényi model cover only parts

More information

Graph models for the Web and the Internet. Elias Koutsoupias University of Athens and UCLA. Crete, July 2003

Graph models for the Web and the Internet. Elias Koutsoupias University of Athens and UCLA. Crete, July 2003 Graph models for the Web and the Internet Elias Koutsoupias University of Athens and UCLA Crete, July 2003 Outline of the lecture Small world phenomenon The shape of the Web graph Searching and navigation

More information

Seminar assignment 1. 1 Part

Seminar assignment 1. 1 Part Seminar assignment 1 Each seminar assignment consists of two parts, one part consisting of 3 5 elementary problems for a maximum of 10 points from each assignment. For the second part consisting of problems

More information

RANKING NATIONAL FOOTBALL LEAGUE TEAMS USING GOOGLE S PAGERANK

RANKING NATIONAL FOOTBALL LEAGUE TEAMS USING GOOGLE S PAGERANK RANKING NATIONAL FOOTBALL LEAGUE TEAMS USING GOOGLE S PAGERANK ANJELA Y. GOVAN AND CARL D. MEYER Abstract. The search engine Google uses the PageRanks of webpages to determine the order in which they are

More information

Graphs over Time Densification Laws, Shrinking Diameters and Possible Explanations

Graphs over Time Densification Laws, Shrinking Diameters and Possible Explanations Graphs over Time Densification Laws, Shrinking Diameters and Possible Explanations Jurij Leskovec, CMU Jon Kleinberg, Cornell Christos Faloutsos, CMU 1 Introduction What can we do with graphs? What patterns

More information

Application of Social Network Analysis to Collaborative Team Formation

Application of Social Network Analysis to Collaborative Team Formation Application of Social Network Analysis to Collaborative Team Formation Michelle Cheatham Kevin Cleereman Information Directorate Information Directorate AFRL AFRL WPAFB, OH 45433 WPAFB, OH 45433 michelle.cheatham@wpafb.af.mil

More information

Routing in the Dark: Pitch Black

Routing in the Dark: Pitch Black Routing in the Dark: Pitch Black Nathan S. Evans Nathan.S.Evans@du.edu Christian Grothoff christian@grothoff.org Colorado Research Institute for Security and Privacy University of Denver 1 Motivation Efficient

More information

Graph Algorithms and Graph Databases. Dr. Daisy Zhe Wang CISE Department University of Florida August 27th 2014

Graph Algorithms and Graph Databases. Dr. Daisy Zhe Wang CISE Department University of Florida August 27th 2014 Graph Algorithms and Graph Databases Dr. Daisy Zhe Wang CISE Department University of Florida August 27th 2014 1 Google Knowledge Graph -- Entities and Relationships 2 Graph Data! Facebook Social Network

More information

Exploring Big Data in Social Networks

Exploring Big Data in Social Networks Exploring Big Data in Social Networks virgilio@dcc.ufmg.br (meira@dcc.ufmg.br) INWEB National Science and Technology Institute for Web Federal University of Minas Gerais - UFMG May 2013 Some thoughts about

More information

Healthcare Analytics. Aryya Gangopadhyay UMBC

Healthcare Analytics. Aryya Gangopadhyay UMBC Healthcare Analytics Aryya Gangopadhyay UMBC Two of many projects Integrated network approach to personalized medicine Multidimensional and multimodal Dynamic Analyze interactions HealthMask Need for sharing

More information

USER INTENT PREDICTION FROM ACCESS LOG IN ONLINE SHOP

USER INTENT PREDICTION FROM ACCESS LOG IN ONLINE SHOP IADIS International Journal on WWW/Internet Vol. 12, No. 1, pp. 52-64 ISSN: 1645-7641 USER INTENT PREDICTION FROM ACCESS LOG IN ONLINE SHOP Hidekazu Yanagimoto. Osaka Prefecture University. 1-1, Gakuen-cho,

More information

Hidden Markov Models. and. Sequential Data

Hidden Markov Models. and. Sequential Data Hidden Markov Models and Sequential Data Sequential Data Often arise through measurement of time series Snowfall measurements on successive days in Buffalo Rainfall measurements in Chirrapunji Daily values

More information

The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity

The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity David Cohn Burning Glass Technologies 201 South Craig St, Suite 2W Pittsburgh, PA 15213 david.cohn@burning-glass.com

More information

Machine Learning over Big Data

Machine Learning over Big Data Machine Learning over Big Presented by Fuhao Zou fuhao@hust.edu.cn Jue 16, 2014 Huazhong University of Science and Technology Contents 1 2 3 4 Role of Machine learning Challenge of Big Analysis Distributed

More information

Google-bombing - Manipulating the PageRank Algorithm

Google-bombing - Manipulating the PageRank Algorithm Google-bombing - Manipulating the PageRank Algorithm Peter A. Hamilton CMSC 676 - Information Retrieval 1. Introduction With the growth of the Internet, the field of Information Retrieval (IR) has gained

More information

Optimizing Web Servers Using Page Rank Prefetching for Clustered Accesses.

Optimizing Web Servers Using Page Rank Prefetching for Clustered Accesses. Optimizing Web Servers Using Page Rank Prefetching for Clustered Accesses. by VICTOR Y SAFRONOV A thesis submitted to the Graduate School-New Brunswick Rutgers, The State University of New Jersey in partial

More information

EVILSEED: A Guided Approach to Finding Malicious Web Pages

EVILSEED: A Guided Approach to Finding Malicious Web Pages + EVILSEED: A Guided Approach to Finding Malicious Web Pages Presented by: Alaa Hassan Supervised by: Dr. Tom Chothia + Outline Introduction Introducing EVILSEED. EVILSEED Architecture. Effectiveness of

More information

Top Online Activities (Jupiter Communications, 2000) CS276A Text Information Retrieval, Mining, and Exploitation

Top Online Activities (Jupiter Communications, 2000) CS276A Text Information Retrieval, Mining, and Exploitation Top Online Activities (Jupiter Communications, 2000) CS276A Text Information Retrieval, Mining, and Exploitation Lecture 11 12 November, 2002 Email Web Search 88% 96% Special thanks to Andrei Broder, IBM

More information

Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

More information

Research Article A Comparison of Online Social Networks and Real-Life Social Networks: A Study of Sina Microblogging

Research Article A Comparison of Online Social Networks and Real-Life Social Networks: A Study of Sina Microblogging Mathematical Problems in Engineering, Article ID 578713, 6 pages http://dx.doi.org/10.1155/2014/578713 Research Article A Comparison of Online Social Networks and Real-Life Social Networks: A Study of

More information

Big Data Technology Motivating NoSQL Databases: Computing Page Importance Metrics at Crawl Time

Big Data Technology Motivating NoSQL Databases: Computing Page Importance Metrics at Crawl Time Big Data Technology Motivating NoSQL Databases: Computing Page Importance Metrics at Crawl Time Edward Bortnikov & Ronny Lempel Yahoo! Labs, Haifa Class Outline Link-based page importance measures Why

More information

Data Mining and Information Retrieval. Introduction to Web Mining

Data Mining and Information Retrieval. Introduction to Web Mining Data Mining and Information Retrieval Introduction to Web Mining What is Web Mining? Discovering useful information from the World-Wide Web and its usage patterns. CMPT 454: Database Systems II Introduction

More information

Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network

Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network , pp.273-284 http://dx.doi.org/10.14257/ijdta.2015.8.5.24 Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network Gengxin Sun 1, Sheng Bin 2 and

More information

Today. Concepts. Semantics and Categorization. Functions of Categories. Bruner, Goodnow, & Austin (1956) Rules

Today. Concepts. Semantics and Categorization. Functions of Categories. Bruner, Goodnow, & Austin (1956) Rules Today Concepts Intro Psychology Georgia Tech Instructor: Dr. Bruce Walker Categories Prototypes Networks Imagery Thinking Functions of Categories Reduce complexity Determine appropriate actions Provides

More information

Small Maximal Independent Sets and Faster Exact Graph Coloring

Small Maximal Independent Sets and Faster Exact Graph Coloring Small Maximal Independent Sets and Faster Exact Graph Coloring David Eppstein Univ. of California, Irvine Dept. of Information and Computer Science The Exact Graph Coloring Problem: Given an undirected

More information

Analysis and Computation of Google s PageRank

Analysis and Computation of Google s PageRank Analysis and Computation of Google s PageRank Ilse Ipsen North Carolina State University Joint work with Rebecca M. Wills IMACS p.1 Overview Goal: Compute (citation) importance of a web page Simple Web

More information

EFFICIENT DETECTION IN DDOS ATTACK FOR TOPOLOGY GRAPH DEPENDENT PERFORMANCE IN PPM LARGE SCALE IPTRACEBACK

EFFICIENT DETECTION IN DDOS ATTACK FOR TOPOLOGY GRAPH DEPENDENT PERFORMANCE IN PPM LARGE SCALE IPTRACEBACK EFFICIENT DETECTION IN DDOS ATTACK FOR TOPOLOGY GRAPH DEPENDENT PERFORMANCE IN PPM LARGE SCALE IPTRACEBACK S.Abarna 1, R.Padmapriya 2 1 Mphil Scholar, 2 Assistant Professor, Department of Computer Science,

More information

Automated Collage Generation With Intent

Automated Collage Generation With Intent Automated Collage Generation With Intent Anna Krzeczkowska, Jad El-Hage, Simon Colton and Stephen Clark Department of Computing, Imperial College, London, UK. University of Cambridge Computer Laboratory,

More information

Trinity: A Distributed Graph Engine on a Memory Cloud

Trinity: A Distributed Graph Engine on a Memory Cloud Trinity: A Distributed Graph Engine on a Memory Cloud Review of: B. Shao, H. Wang, Y. Li. Trinity: a distributed graph engine on a memory cloud, Proc. ACM SIGMOD International Conference on Management

More information

Music Lyrics Spider. Wang Litan 1 and Kan Min Yen 2 School of Computing, National University of Singapore 3 Science Drive 2, Singapore ABSTRACT

Music Lyrics Spider. Wang Litan 1 and Kan Min Yen 2 School of Computing, National University of Singapore 3 Science Drive 2, Singapore ABSTRACT NUROP CONGRESS PAPER Music Lyrics Spider Wang Litan 1 and Kan Min Yen 2 School of Computing, National University of Singapore 3 Science Drive 2, Singapore 117543 ABSTRACT The World Wide Web s dynamic and

More information

NETZCOPE - a tool to analyze and display complex R&D collaboration networks

NETZCOPE - a tool to analyze and display complex R&D collaboration networks The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots NETZCOPE - a tool to analyze and display complex R&D collaboration networks L. Streit & O. Strogan BiBoS, Univ.

More information

Search Engines. Stephen Shaw <stesh@netsoc.tcd.ie> 18th of February, 2014. Netsoc

Search Engines. Stephen Shaw <stesh@netsoc.tcd.ie> 18th of February, 2014. Netsoc Search Engines Stephen Shaw Netsoc 18th of February, 2014 Me M.Sc. Artificial Intelligence, University of Edinburgh Would recommend B.A. (Mod.) Computer Science, Linguistics, French,

More information

Predicting Influentials in Online Social Networks

Predicting Influentials in Online Social Networks Predicting Influentials in Online Social Networks Rumi Ghosh Kristina Lerman USC Information Sciences Institute WHO is IMPORTANT? Characteristics Topology Dynamic Processes /Nature of flow What are the

More information

DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II. Matrix Algorithms DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

More information

Complex Network Visualization based on Voronoi Diagram and Smoothed-particle Hydrodynamics

Complex Network Visualization based on Voronoi Diagram and Smoothed-particle Hydrodynamics Complex Network Visualization based on Voronoi Diagram and Smoothed-particle Hydrodynamics Zhao Wenbin 1, Zhao Zhengxu 2 1 School of Instrument Science and Engineering, Southeast University, Nanjing, Jiangsu

More information

princeton university F 02 cos 597D: a theorist s toolkit Lecture 7: Markov Chains and Random Walks

princeton university F 02 cos 597D: a theorist s toolkit Lecture 7: Markov Chains and Random Walks princeton university F 02 cos 597D: a theorist s toolkit Lecture 7: Markov Chains and Random Walks Lecturer: Sanjeev Arora Scribe:Elena Nabieva 1 Basics A Markov chain is a discrete-time stochastic process

More information

Complex Networks Analysis: Clustering Methods

Complex Networks Analysis: Clustering Methods Complex Networks Analysis: Clustering Methods Nikolai Nefedov Spring 2013 ISI ETH Zurich nefedov@isi.ee.ethz.ch 1 Outline Purpose to give an overview of modern graph-clustering methods and their applications

More information

USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS

USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE- FREE NETWORKS AND SMALL-WORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu

More information

Algorithms for representing network centrality, groups and density and clustered graph representation

Algorithms for representing network centrality, groups and density and clustered graph representation COSIN IST 2001 33555 COevolution and Self-organization In dynamical Networks Algorithms for representing network centrality, groups and density and clustered graph representation Deliverable Number: D06

More information

A Graph theoretical approach to Network Vulnerability Analysis and Countermeasures

A Graph theoretical approach to Network Vulnerability Analysis and Countermeasures A Graph theoretical approach to Network Vulnerability Analysis and Countermeasures Dr.Thaier Hamid University of Bedfordshire, UK Prof. Carsten Maple, University of Bedfordshire, UK ABSTRACT Computer networks

More information

Collecting Network Data in Surveys

Collecting Network Data in Surveys Collecting Network Data in Surveys Arun Advani and Bansi Malde September 203 We gratefully acknowledge funding from the ESRC-NCRM Node Programme Evaluation for Policy Analysis Grant reference RES-576-25-0042.

More information

CS 207 - Data Science and Visualization Spring 2016

CS 207 - Data Science and Visualization Spring 2016 CS 207 - Data Science and Visualization Spring 2016 Professor: Sorelle Friedler sorelle@cs.haverford.edu An introduction to techniques for the automated and human-assisted analysis of data sets. These

More information

Big Data Analytics. Lucas Rego Drumond

Big Data Analytics. Lucas Rego Drumond Big Data Analytics Lucas Rego Drumond Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany MapReduce II MapReduce II 1 / 33 Outline 1. Introduction

More information

RANKING WEB PAGES RELEVANT TO SEARCH KEYWORDS

RANKING WEB PAGES RELEVANT TO SEARCH KEYWORDS ISBN: 978-972-8924-93-5 2009 IADIS RANKING WEB PAGES RELEVANT TO SEARCH KEYWORDS Ben Choi & Sumit Tyagi Computer Science, Louisiana Tech University, USA ABSTRACT In this paper we propose new methods for

More information

SGL: Stata graph library for network analysis

SGL: Stata graph library for network analysis SGL: Stata graph library for network analysis Hirotaka Miura Federal Reserve Bank of San Francisco Stata Conference Chicago 2011 The views presented here are my own and do not necessarily represent the

More information

Efficient Search in Gnutella-like Small-World Peerto-Peer

Efficient Search in Gnutella-like Small-World Peerto-Peer Efficient Search in Gnutella-like Small-World Peerto-Peer Systems * Dongsheng Li, Xicheng Lu, Yijie Wang, Nong Xiao School of Computer, National University of Defense Technology, 410073 Changsha, China

More information

Studying E-mail Graphs for Intelligence Monitoring and Analysis in the Absence of Semantic Information

Studying E-mail Graphs for Intelligence Monitoring and Analysis in the Absence of Semantic Information Studying E-mail Graphs for Intelligence Monitoring and Analysis in the Absence of Semantic Information Petros Drineas, Mukkai S. Krishnamoorthy, Michael D. Sofka Bülent Yener Department of Computer Science,

More information

The Second Eigenvalue of the Google Matrix

The Second Eigenvalue of the Google Matrix 0 2 The Second Eigenvalue of the Google Matrix Taher H Haveliwala and Sepandar D Kamvar Stanford University taherh,sdkamvar @csstanfordedu Abstract We determine analytically the modulus of the second eigenvalue

More information

Big Data Systems CS 5965/6965 FALL 2015

Big Data Systems CS 5965/6965 FALL 2015 Big Data Systems CS 5965/6965 FALL 2015 Today General course overview Expectations from this course Q&A Introduction to Big Data Assignment #1 General Course Information Course Web Page http://www.cs.utah.edu/~hari/teaching/fall2015.html

More information

Social Media Mining. Graph Essentials

Social Media Mining. Graph Essentials Graph Essentials Graph Basics Measures Graph and Essentials Metrics 2 2 Nodes and Edges A network is a graph nodes, actors, or vertices (plural of vertex) Connections, edges or ties Edge Node Measures

More information

PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS.

PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS. PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS Project Project Title Area of Abstract No Specialization 1. Software

More information

Part 2: Community Detection

Part 2: Community Detection Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

More information