NETZCOPE - a tool to analyze and display complex R&D collaboration networks

Transcription

1 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots NETZCOPE - a tool to analyze and display complex R&D collaboration networks L. Streit & O. Strogan BiBoS, Univ. Bielefeld and CCM, Univ. da Madeira ZiF Conference Complexity, Mathematics and Socio-Economic Problems August 31 - September 12, 2009 L. Streit & O. Strogan Complex Networks

2 Outline The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots 1 The Task 2 Concepts from Spectral Graph Theory 3 EU R&D Network Analysis 4 Netzcope Screenshots L. Streit Complex Networks

3 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Collaborative R&D in the EU Framework Programmes The empirical infrastructure sysres EUPRO Framework Programme (FP) Period Projects Projects > 1 partner Organisations Subentities FP FP FP FP FP FP Total Based on information from CORDIS, with major standardisation and cleaning process applied Current release January 2008 Covering FP 1 to FP 6 systems research L. Streit Complex Networks

4 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots The consortium NEMO partners and website CCM, Universidade da Madeira systems research L. Streit Complex Networks

5 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots NEMO Tasks L. Streit Complex Networks

6 The Introduction Task Concepts Concepts from Spectral from Spectral Graph Graph Theory Theory EU R&D EU R&D Network Network Analysis Analysis Netzcope Screenshots The State Sizeof of the the art Problem Weighted Graphs Spectral Gaps Comparison of Centralities The Effect of Fiedler Ordering Visualizing the EU Research Networks L. Streit with T. Krueger Spectral Properties of a Real World Network l L. Streit Complex Networks

7 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Adjacency Matrix A simple graph G = (V,E) is descibed by an adjacency matrix indicating whether verices i and k are connected by an edge: { 1 i k a ik = 0 otherwise The degree of edge k is and we shall set d k = a ik i D = diag {d 1,...,d n } L. Streit Complex Networks

8 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Bipartite Graphs Two types of vertices V = {V 1,V 2 } No edges between vertices of the same type. L. Streit Complex Networks

9 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots The Laplacian(s) The Laplacian and the normalized Laplacian L = D A L = 1 D 1/2 AD 1/2 play a central role, in particular L is (up to a similarity transformation) the generator of a continuous time random walk on the graph, with equal probability 1/d k along each edge. L. Streit Complex Networks

10 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Eigenvalues of L Have 0 = λ 0 λ 1...λ n 2 The multiplicity of λ 0 equals the number of disconnected subgraphs in G. Fully connected graphs with n vertices are characterized by λ 1 =... = λ n = Bipartite graphs are characterized by λ n = 2 n n 1 and have a symmetric spectrum: with λ k, 2 λ n is also an eigenvalue. L. Streit Complex Networks

11 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots A Variational Formula Consider the map on R n x ˆx = D 1/2 x Have (x,l x) = 1 2 a ik (ˆx i ˆx k ) 2 Hence one eigenvector, corresponding to the minimal eigenvalue λ 0 = 0, is given by ê k 1. L. Streit & O. Strogan Complex Networks

12 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots I.e. and hence Theorem e k = d 1/2 k The invariant distribution for the random walk is p k = d k d k L. Streit Complex Networks

13 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Keeping the Little Guy Waiting (Previously r B A = 1, r A B = 1/3) ( 1 Now : r A B = r B A = 1/3 = min, 1 ) d A d B L. Streit Complex Networks

14 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots The Laplacian For this random walk with waiting, the normalized Laplacian will be ) min( 1 d i L ik =, 1 d k i k L ik i = k k:k i L. Streit Complex Networks

15 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Spectral Gap For a connected graph the spectral gap λ 1 measures the rate of knowledge diffusion through the network. Note that λ 1 = min (x,l x) Definition x e, x =1 The minimizing vector f is called the Fiedler vector (1). L. Streit Complex Networks

16 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots The Fiedler Vector (2, normalized) Recall (f,l f ) = 1 2 a ik (ˆfi ˆf k ) 2 Imagine having arranged the vertices of the graph at points y k on the real line, attracting each other by an elastic force acting between them along each edge. Then the total energy of the configuration is E = 1 2 a ik (y i y k ) 2 We recognize the positions y k = ˆf k as the configuration of minimal energy, subject to the constraints y = D 1/2 ŷ = 1, y e which fix the extension of the configuration and its localization L. Streit Complex Networks

17 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Let us say that the centrality c k of vertex k is proportional to the centrality of all its neighbors in the network: c k = 1 λ a kl c l. (A web page, e.g., is important if it is linked to other important pages.) In other words, Ac = λc By Perron-Frobenius, there is a solution with non negative c k, corresponding to the largest eigenvalue of the adjacency matrix A. Definition The resulting c k are called eigenvalue centralities. L. Streit Complex Networks

18 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Newman s Modularity "A good division of a network into communities is not merely one in which there are few edges between communities; it is one in which there are fewer than expected edges between communities". Newman, Modularity is - up to a normalization constant - the number of edges within communities c minus those for a null model: Q 1 2 E c (A ij P ij ). i,j c L. Streit Complex Networks

19 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots The Null Model The standard choice for the null model will be dictated by the degrees of the vertices i and j: P ij d id j 2 E corresponding to a random graph models with a fixed set of vertices and the constraint that on average they should reproduce a given degree distribution (Chung,Lu 2002). The goal now is to find a division of the vertices into communities such that the modularity Q is maximal. An exhaustive search for a decomposition is out of the question: even for moderately big graphs there are far too many ways to decompose them into communities; fast approximate algorithms do exist. L. Streit Complex Networks

20 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Barber s BRIM For bipartite graphs the null model should be modified, to reproduce the characteristic form of bipartite adjacency matrices ( ) 0 B A = B T 0 also for the null model. Recently M. Barber (2007) proposed an appropriate algorithm ("BRIM": bipartite, recursively induced modules) to find communities for bipartite networks. Starting from a (more or less) ad hoc partition of the vertices of type 1 it is straightforward to optimize a corresponding decomposition of the verices of type 2. From there, optimize the decomposition of vertices of type 1, and iterate. In this fashion, modularity will increase until a (local) maximum is reached. L. Streit Complex Networks

21

22

23

24

25

26

27

28

29

30 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Organization Graph Spectra FP1, FP2, FP3 L. Streit Complex Networks

31 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots A Simulated Network L. Streit Complex Networks

32 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Weighted vs. Unweighted, FP 2 L. Streit Complex Networks

33 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Spectrum with the Little Guy waiting L. Streit Complex Networks

34

35

36

37

38

39

40

41

42

43

44 Zooming in

45

46

47

48

49

50

51

52 A Small Community

53 Geographic Profile of a Community

54 Organization Profile

55 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots Summary of Netzcope Features Netzcope extracts the connected components produces 2D plot of the networks displays the adjacency matrix same after re-ordering the nodes w.r.t. their strength of interaction finds communities within the network following Newman uses the BRIM algorithm to enhance the modularity computes mutual information of different decompositions produces a scatter plot to identify central players and gate keepers displays the composition of communities exhibits degree and eigenvalue centralities within communities displays the graph portrait following Bagrow et al. permits iterative decomposition and analysis ( zooming into communities ) L. Streit Complex Networks

56 The Task Concepts from Spectral Graph Theory EU R&D Network Analysis Netzcope Screenshots M. J. Barber: Modularity and community detection in bipartite networks. Phys. Rev. E 76, (2007). F. Chung and L. Lu: Connected components in random graphs with given degree sequences. Annals of Combinatorics 6, (2002). M. E. J. Newman and M. Girvan: Finding and evaluating community structure in networks. Phys. Rev. E 69, (2004). M. E. J. Newman: Finding community structure in networks using the eigenvectors of matrices. arxiv:physics/ v3 23 Jul 2006 M. E. J. Newman: Modularity and community structure in networks. URL L. Streit Complex Networks