Big Data: Opportunities and Challenges for Complex Networks

Transcription

1 Big Data: Opportunities and Challenges for Complex Networks Jinhu Lü Academy of Mathematics and Systems Science Chinese Academy of Sciences IWCSN, Vancouver, 13 Dec

2 Thanks To Prof. Ljiljana j Trajkovic Simon Fraser University, Canada Prof. Guanrong Chen City University of Hong Kong, China Prof. David Hill The University of Hong Kong, China Prof. Sherman Shen University of Waterloo, Canada Prof. Benjamin W. Wah The Chinese University of HK, China 2

3 Outline Complex Networks vs. Big Data Big Data: Opportunities and Challenges Real-World Applications: Smart Grid Theoretical Advances: Analysis and Control Conclusion 3

5 Complex Networks vs. Big Data Traditionally, complex networks are studied via Graph Theory - Erdös and Rényi (1960) ER Random Graphs ER Random Graph model dominates for 50 years till today Availability of Big Data and Supper-Fast computing power have led to a Rethinking of the above approach Two significant ifi recent tdiscoveries i are: Small-World effect (e.g., Watts and Strogatz, 1998) Scale-Free feature (e.g., Barabási and Albert, 1999)

6 Outline Introduction to Complex Networks Big Data: Opportunities and Challenges Real-World Applications: Smart Grid Theoretical Advances: Analysis and Control Conclusion 6

7 Gartner Hype Cycle (Special Report: July 27, 2013) 7

8 Big Data and the 4 Vs Volume Velocity Variety Veracity Very Large Volume Very Fast Volume Multimodal l True or False Volume 文本图片 Volume 视频到 2020 年, 数据总量分享的内容条目超过达 40ZB, 人均 5.2TB 25 亿个 / 天, 增加数据超过 500TB/ 天音频 8

9 US Big Data Initiative US $200m on March 29, Federal departments and agencies NSF, HHS/NIH, DOD, DOE, DARPA, USGS Some initiatives BRAIN (Brain Research through Advancing Innovative Neuro-technologies) Open Science Data Cloud (NSF) DiD: Digging into Data Challenge in SSc & humanities Big Data-Aware Terabits Networking (DOE) NEX: NASA Earth Exchange PRISM (NSA) 9

10 Some European Efforts The European Commission 2-year-long Big Data Public Private Forum through their Seventh Framework Program to engage companies, academics and other stakeholders in discussing Big Data issues. Define a research and innovation strategy to guide a successful implementation of Big Data economy. Outcomes to be used as input for Horizon 2020, their next framework program CERN Open Lab 10

11 Big-Data in China Internet and Big Data (973 Project) Network Communication and Big Data (NSF) Cyberspace Security and Big Data (973 Project) Urbanization, ation Smart City (40 Billion) Finance and Big Data Health Care and Big Data (863 Project) Materials, Manufacturing and Big Data Bioinformatics, i Pharmaceutical and Big Data 11

12 Making Big Data into Small Data Final Abstraction Further Abstraction ti Abstracted Networks Network of Big Data 12

14 Smart Grid 14

15 Smart Grid in China 15

16 Current Power Grid Power Flow Four Main Domains of the Power System/Grid Power generation / Power transmission production Power consumption / Power distribution load 16

17 What is Smart Grid? 17

18 Benefit of Smart Grid(1) Benefits of Smart Grid in United States (Source: IEEE Smart Grid) The cost of nationwide smart grid ranges around $340 billion to $480 billion, over a 20-year period, which is equivalent to $20-$25 billion per year Right off the bat, the benefits are $70 billion per year in reduced costs from outages. On a year with hurricanes, ice storms, and/or other disturbances, the benefits would even be higher The benefits include reducing the costs of outages by about $49 billion per year, andreducing CO 2 emissions by 12-18% by 2030 The benefits e also asoinclude increasing system efficiency e cy by over 4percent which is about $20.4 billion per year 18

19 Benefit of Smart Grid (2) Benefits of a Campus-wide Microgrid (Source: IIT) System cost: $12 million Reliability is improved such that 3-4 power outages per year are avoided. Previously, power failures cost IIT about $1 million per year $500,000 to $1.5 million per year is saved by improving system efficiency and reducing electricity usage and electricity peak demand $7 million is saved from avoided infrastructure (substation) upgrades Illinois Institute of Technology (IIT) Perfect Power Microgrid 19

21 Case I: Control of fcomplex Networks [1] J. Zhou, J. Lu, J. Lü, Automatica, 44: , [2] W. Yu, G. Chen, J. Lü, Automatica, 45: , Y. [3] Chen, J. Lü, X. Yu, Z. Lin, SIAM J. Contr. Optim., 51(4): , [4] W. Yu, G. Chen, J. Lü, J. Kurths, SIAM J. Contr. Optim., 51(2): ,

22 Control of Complex Networks Discovery: What do real networks look like even if we can t actually look at them? Modeling: How to model them? Impact: How does the topology of a network affect its function? Control: How can the topological lcharacteristics ti be used to improve the function of a network? 22

23 Control of Complex Networks Feasibility:Can i i the goal of control be achieved by only directly control a fraction of nodes? Control Science Efficiency:Howtoselect the nodes to be controlled so that the goal can be achieved with a low cost? Network Science Focus: complexity of the network structure 23

24 Control of Complex Networks 24

25 Pinning Control of Complex Networks For a large complex networks, it is often impossible to control every node. Is it possible to control asmall fraction of nodes (e.g. 5%) to achieve the same effect? Pinning Control 25

26 Pinning Control: An Example 26

27 Key Factors of Pinning Control Network Structure Regular, random, power-law, small-world, Coupling Strength th Strong, weak, Number of Controllers (and, what type of controllers?) Control Strategies Random, selective, Trade-off? 27

28 Detecting Community Structure: Challenges Network Evolution and Emergence 28

29 Two Fundamental Problems How many and which nodes should a network with fixed structure and coupling strength be pinned to reach network synchronization? How large the coupling strength should a network with fixed structure and pinning nodes be applied to reach network synchronization? J. Zhou, J. Lu, J. Lü, Automatica, 44: , W. Yu, G. Chen, J. Lü, Automatica, 45: , 435,

30 Some Main Advances A simply approximate formula is deduced for estimating the detailed number of pinning i nodes and the magnitude of the coupling strength for a given general complex dynamical network Y. Chen,J.Lü,X.Yu,Z.Lin,SIAM J. Contr. Optim., 51(4): , W. Yu, G. Chen, J. Lü, J. Kurths, SIAM J. Contr. Optim., 51(2): ,

31 Case II: When Structure Meets Function in Evolutionary Dynamics on Complex Networks [1] S. Tan, J. Lü, G. Chen, D. Hill, When structure meets function in evolutionary dynamics on complex networks, IEEE Circuits Syst. Mag., in press, [2]S.Tan,J.Lü,X.Yu,D.Hill,Chin. Sci. Bull., 58(28-29): , [3] S. Tan, J. Lü, D. Hill, Towards a theoretical framework for controlling random drift on complex networks, IEEE Trans. Auto. Contr., revision,

32 Background and Motivation It is well known that the genotypes, phenotypes, and behaviors of population are evolving with time There are three fundamental principles i of evolution: Reproduction, Mutation and Selection One interesting question is what the population looks like in the end. In other words, which h type of individual survives and which is eliminated

33 Evolutionary Dynamics Evolutionary dynamics models the evolution processes of population. p Two fundamental models are the Moran Process and Wright- Fisher Process Generally, a population with constant size N and two types of individuals Mutants (M) and Residents (R) is assumed

34 Population Structure Unconnected populations Connected but not strong connected populations Nodes representing the individuals and links representing the interaction, structured population is modeled as network Much interest focuses on the general structured population

35 Fitness of Individual Constant Selection: The fitness of mutant is set as r and the resident is set as 1 Frequency-Dependent Selection: Every individual plays a two-by-two game with all neighbors, and the fitness is determined by its payoff

36 Updating Rules (A) Birth-Death thprocess At each step with a probability proportional to its fitness An individual is selected from the population to reproduce The offspring replaces a random chosen neighbor

37 Updating Rules (B) Death-Birth Process At each step, a random individual is selected to die All neighbors compete for the above location With a probability proportional to its fitness, a neighbor wins the location and leaves an offspring there

38 Updating Rules (C) Link k Dynamics: At each step, one link (i, j) is selected randomly, and then individual i reproduces and dits offspring takes over individual j Other Dynamics: Imitation Dynamics Wright-Fisher Process Invasion Process

39 Updating Rules: Examples

40 Evolutionary Dynamics Initially Mutation Evolution Evolution Ends 第 40 页

41 Evolutionary Dynamics Evolution Extinction Residents & Mutants Initially Middle Process Evolution Final Status Fixation 第 41 页

42 Evolutionary Dynamics on Network

43 Description of Problem AL Long-Standing Open Problem The individual with a higher fitness will have a higher survival probability Essential Difficulty: Computational complexity of fixation probability

44 Main Aim It aims at providing a rigorously theoretical proof for the global existence of such property in the locall evolutionary dynamics by using the coupling and splicing techniques We also prove that the fixation probability is monotone increasing for the initial nodes set of mutants

45 Fitness Setting Random Drifts: fitness=1 fitness=1 Constant Selection: fitness=1 fitness=r Evolutionary o Game Dynamics: a b fitness=a*n( )+ b*n( ) c d fitness=c*n( )+ d*n( )

46 Fixation Probability Problem Fixation Probability Problem: The probability that the mutants eventually spread and take over the whole population

47 Mathematical Description (A) Survival i lof fthe Fittest t is the stone principle i in population evolution Mathematically, the following equation should hold: ( M, r ) (, ) r r 1 M r 2 if 1 2

48 Mathematical Description (B) The more mutants there are initially, i i the more likely the mutants fixed in the end Mathematically, the following equation should hold: ( M, r) ( T, r) M T if

49 Case A:WellMixed Well-Mixed Population In well-mixed population, for the initially mutants set M and fitness r, the fixation probability can be easily derived: ( M, r) 1 r 1 1 N r 1 M where N is the population size

50 Case B: Structured Population In structured population, the computation of fixation probability has very high complexity for complex networks How to prove the two basic properties p for the evolutionary dynamics with structured populations without computing the fixation probability?

51 Main Results Theorem 1: In evolutionary dynamics on arbitrary connected networks with death- birth updating rule, the fixation probability ρ(m,r) is monotone increasing with the initial mutant set M, i.e., ( M, r) ( T, r) if M T

52 Main Results Theorem 2: In evolutionary dynamics on arbitrary connected networks with death- birth updating rule, the fixation probability ρ(m,r) is monotone increasing with the mutant s fitness r, i.e., ( M, r) (, ) 1 M r2 if r1 r2

53 Numerical Simulations A random geometric network with 20 individuals id

54 Numerical Analysis (A) For different fitness, the fixation probability increases, p y monotonically with the size of mutants set

55 Numerical Analysis (B) For different mutant set, the fixation probability, p y increases monotonically with the fitness of mutant r

56 Outline Introduction to Complex Networks Big Data: Opportunities and Challenges Real-World Applications: Smart Grid Theoretical Advances: Analysis and Control Conclusion 56

57 Conclusion (Future Research ) Driven by the Major National Strategic Needs Smart Grid Internet of Things Mobile Internet Location Based Services Gene Regulatory Networks Driven by the Forefront of International Academics Analysis and Control Key Fundamental Issues 57

58 58