CMU SCS Mining Billion-Node Graphs - Patterns and Algorithms
|
|
- David Willis
- 8 years ago
- Views:
Transcription
1 Mining Billion-Node Graphs - Patterns and Algorithms Christos Faloutsos CMU
2 Thank you! Panos Chrysanthis Ling Liu Vladimir Zadorozhny Prashant Krishnamurthy C. Faloutsos (CMU) 2
3 Resource Open source system for mining huge graphs: PEGASUS project (PEta GrAph mining System) code and papers C. Faloutsos (CMU) 3
4 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Problem#3: Scalability Conclusions C. Faloutsos (CMU) 4
5 Graphs - why should we care? >$10B revenue Food Web [Martinez 91] >0.5B users Internet Map [lumeta.com] C. Faloutsos (CMU) 5
6 Graphs - why should we care? IR: bi-partite graphs (doc-terms) D T 1 web: hyper-text graph D N T M... and more: C. Faloutsos (CMU) 6
7 Graphs - why should we care? viral marketing web-log ( blog ) news propagation computer network security: /ip traffic and anomaly detection... Subject-verb-object -> graph Many-to-many db relationship -> graph C. Faloutsos (CMU) 7
8 Outline Introduction Motivation Problem#1: Patterns in graphs Static graphs Weighted graphs Time evolving graphs Problem#2: Tools Problem#3: Scalability Conclusions C. Faloutsos (CMU) 8
9 Problem #1 - network and graph mining What does the Internet look like? What does FaceBook look like? What is normal / abnormal? which patterns/laws hold? C. Faloutsos (CMU) 9
10 Problem #1 - network and graph mining What does the Internet look like? What does FaceBook look like? What is normal / abnormal? which patterns/laws hold? To spot anomalies (rarities), we have to discover patterns C. Faloutsos (CMU) 10
11 Problem #1 - network and graph mining What does the Internet look like? What does FaceBook look like? What is normal / abnormal? which patterns/laws hold? To spot anomalies (rarities), we have to discover patterns Large datasets reveal patterns/anomalies that may be invisible otherwise C. Faloutsos (CMU) 11
12 Graph mining Are real graphs random? C. Faloutsos (CMU) 12
13 Laws and patterns Are real graphs random? A: NO!! Diameter in- and out- degree distributions other (surprising) patterns So, let s look at the data C. Faloutsos (CMU) 13
14 Solution# S.1 Power law in the degree distribution [SIGCOMM99] internet domains log(degree) att.com ibm.com log(rank) C. Faloutsos (CMU) 14
15 Solution# S.1 Power law in the degree distribution [SIGCOMM99] internet domains log(degree) att.com ibm.com log(rank) C. Faloutsos (CMU) 15
16 Solution# S.2: Eigen Exponent E Eigenvalue Exponent = slope E = May 2001 Rank of decreasing eigenvalue A2: power law in the eigenvalues of the adjacency matrix C. Faloutsos (CMU) 16
17 Solution# S.2: Eigen Exponent E Eigenvalue Exponent = slope E = May 2001 Rank of decreasing eigenvalue [Mihail, Papadimitriou 02]: slope is ½ of rank exponent C. Faloutsos (CMU) 17
18 But: How about graphs from other domains? C. Faloutsos (CMU) 18
19 More power laws: web hit counts [w/ A. Montgomery] Web Site Traffic Count (log scale) Zipf ``ebay users sites in-degree (log scale) C. Faloutsos (CMU) 19
20 epinions.com count who-trusts-whom [Richardson + Domingos, KDD 2001] trusts-2000-people user (out) degree C. Faloutsos (CMU) 20
21 And numerous more # of sexual contacts Income [Pareto] distribution Duration of downloads [Bestavros+] Duration of UNIX jobs ( mice and elephants ) Size of files of a user Black swans C. Faloutsos (CMU) 21
22 Roadmap Introduction Motivation Problem#1: Patterns in graphs Static graphs degree, diameter, eigen, triangles cliques Weighted graphs Time evolving graphs Problem#2: Tools C. Faloutsos (CMU) 22
23 Solution# S.3: Triangle Laws Real social networks have a lot of triangles C. Faloutsos (CMU) 23
24 Solution# S.3: Triangle Laws Real social networks have a lot of triangles Friends of friends are friends Any patterns? C. Faloutsos (CMU) 24
25 Triangle Law: #S.3 [Tsourakakis ICDM 2008] HEP-TH ASN Epinions X-axis: # of participating triangles Y: count (~ pdf) C. Faloutsos (CMU) 25
26 Triangle Law: #S.3 [Tsourakakis ICDM 2008] HEP-TH ASN Epinions X-axis: # of participating triangles Y: count (~ pdf) C. Faloutsos (CMU) 26
27 Triangle Law: #S.4 [Tsourakakis ICDM 2008] Reuters SN Epinions X-axis: degree Y-axis: mean # triangles n friends -> ~n 1.6 triangles C. Faloutsos (CMU) 27
28 details Triangle Law: Computations [Tsourakakis ICDM 2008] But: triangles are expensive to compute (3-way join; several approx. algos) Q: Can we do that quickly? C. Faloutsos (CMU) 28
29 details Triangle Law: Computations [Tsourakakis ICDM 2008] But: triangles are expensive to compute (3-way join; several approx. algos) Q: Can we do that quickly? A: Yes! #triangles = 1/6 Sum ( λ i 3 ) (and, because of skewness (S2), we only need the top few eigenvalues! C. Faloutsos (CMU) 29
30 details Triangle Law: Computations [Tsourakakis ICDM 2008] 1000x+ speed-up, >90% accuracy C. Faloutsos (CMU) 30
31 Triangle counting for large graphs? Anomalous nodes in Twitter(~ 3 billion edges) [U Kang, Brendan Meeder, +, PAKDD 11] C. Faloutsos (CMU) 31
32 Triangle counting for large graphs? Anomalous nodes in Twitter(~ 3 billion edges) [U Kang, Brendan Meeder, +, PAKDD 11] C. Faloutsos (CMU) 32
33 Triangle counting for large graphs? Anomalous nodes in Twitter(~ 3 billion edges) [U Kang, Brendan Meeder, +, PAKDD 11] C. Faloutsos (CMU) 33
34 Triangle counting for large graphs? Anomalous nodes in Twitter(~ 3 billion edges) [U Kang, Brendan Meeder, +, PAKDD 11] C. Faloutsos (CMU) 34
35 EigenSpokes B. Aditya Prakash, Mukund Seshadri, Ashwin Sridharan, Sridhar Machiraju and Christos Faloutsos: EigenSpokes: Surprising Patterns and Scalable Community Chipping in Large Graphs, PAKDD 2010, Hyderabad, India, June C. Faloutsos (CMU) 35
36 EigenSpokes Eigenvectors of adjacency matrix! equivalent to singular vectors (symmetric, undirected graph) C. Faloutsos (CMU) 36
37 EigenSpokes details Eigenvectors of adjacency matrix! equivalent to singular vectors (symmetric, undirected graph) N N C. Faloutsos (CMU) 37
38 EigenSpokes details Eigenvectors of adjacency matrix! equivalent to singular vectors (symmetric, undirected graph) N N C. Faloutsos (CMU) 38
39 EigenSpokes details Eigenvectors of adjacency matrix! equivalent to singular vectors (symmetric, undirected graph) N N C. Faloutsos (CMU) 39
40 EigenSpokes details Eigenvectors of adjacency matrix! equivalent to singular vectors (symmetric, undirected graph) N N C. Faloutsos (CMU) 40
41 EE plot: Scatter plot of scores of u1 vs u2 One would expect Many origin A few scattered ~randomly EigenSpokes 2 nd Principal component u2 u1 1 st Principal component C. Faloutsos (CMU) 41
42 EigenSpokes EE plot: Scatter plot of scores of u1 vs u2 One would expect Many origin A few scattered ~randomly u2 90 o u1 C. Faloutsos (CMU) 42
43 EigenSpokes - pervasiveness Present in mobile social graph! across time and space Patent citation graph C. Faloutsos (CMU) 43
44 EigenSpokes - explanation Near-cliques, or nearbipartite-cores, loosely connected C. Faloutsos (CMU) 44
45 EigenSpokes - explanation Near-cliques, or nearbipartite-cores, loosely connected C. Faloutsos (CMU) 45
46 EigenSpokes - explanation Near-cliques, or nearbipartite-cores, loosely connected C. Faloutsos (CMU) 46
47 EigenSpokes - explanation Near-cliques, or nearbipartite-cores, loosely connected spy plot of top 20 nodes So what?! Extract nodes with high scores! high connectivity! Good communities C. Faloutsos (CMU) 47
48 Bipartite Communities! patents from same inventor(s) `cut-and-paste bibliography! magnified bipartite community C. Faloutsos (CMU) 48
49 (maybe, botnets?) Victim IPs? Botnet members? C. Faloutsos (CMU) 49
50 Roadmap Introduction Motivation Problem#1: Patterns in graphs Static graphs degree, diameter, eigen, triangles cliques Weighted graphs Time evolving graphs Problem#2: Tools C. Faloutsos (CMU) 50
51 Observations on weighted graphs? A: yes - even more laws! M. McGlohon, L. Akoglu, and C. Faloutsos Weighted Graphs and Disconnected Components: Patterns and a Generator. SIG-KDD 2008 C. Faloutsos (CMU) 51
52 Observation W.1: Fortification Q: How do the weights of nodes relate to degree? C. Faloutsos (CMU) 52
53 Observation W.1: Fortification More donors, more $? $10 Reagan $5 $7 Clinton C. Faloutsos (CMU) 53
54 Observation W.1: fortification: Snapshot Power Law Weight: super-linear on in-degree exponent iw : 1.01 < iw < 1.26 More donors, even more $ $10 In-weights ($) Orgs-Candidates e.g. John Kerry, $10M received, from 1K donors $5 Edges (# donors) C. Faloutsos (CMU) 54
55 Roadmap Introduction Motivation Problem#1: Patterns in graphs Static graphs Weighted graphs Time evolving graphs Problem#2: Tools C. Faloutsos (CMU) 55
56 Problem: Time evolution with Jure Leskovec (CMU -> Stanford) and Jon Kleinberg (Cornell CMU) C. Faloutsos (CMU) 56
57 T.1 Evolution of the Diameter Prior work on Power Law graphs hints at slowly growing diameter: diameter ~ O(log N) diameter ~ O(log log N) What is happening in real data? C. Faloutsos (CMU) 57
58 T.1 Evolution of the Diameter Prior work on Power Law graphs hints at slowly growing diameter: diameter ~ O(log N) diameter ~ O(log log N) What is happening in real data? Diameter shrinks over time C. Faloutsos (CMU) 58
59 T.1 Diameter Patents Patent citation network 25 years of 2.9 M nodes 16.5 M edges diameter time [years] C. Faloutsos (CMU) 59
60 T.2 Temporal Evolution of the Graphs N(t) nodes at time t E(t) edges at time t Suppose that N(t+1) = 2 * N(t) Q: what is your guess for E(t+1) =? 2 * E(t) C. Faloutsos (CMU) 60
61 T.2 Temporal Evolution of the Graphs N(t) nodes at time t E(t) edges at time t Suppose that N(t+1) = 2 * N(t) Q: what is your guess for E(t+1) =? 2 * E(t) A: over-doubled! But obeying the ``Densification Power Law C. Faloutsos (CMU) 61
62 T.2 Densification Patent Citations Citations among patents 2.9 M nodes 16.5 M edges Each year is a datapoint E(t) 1.66 N(t) C. Faloutsos (CMU) 62
63 Roadmap Introduction Motivation Problem#1: Patterns in graphs Static graphs Weighted graphs Time evolving graphs Problem#2: Tools C. Faloutsos (CMU) 63
64 T.3 : popularity over time # in links lag: days after post Post popularity + lag C. Faloutsos (CMU) 64
65 T.3 : popularity over time # in links (log) Post popularity drops-off exponentially? POWER LAW! Exponent? days after post (log) C. Faloutsos (CMU) 65
66 T.3 : popularity over time # in links (log) -1.6 Post popularity drops-off exponentially? POWER LAW! Exponent? -1.6 close to -1.5: Barabasi s stack model and like the zero-crossings of a random walk days after post (log) C. Faloutsos (CMU) 66
67 -1.5 slope J. G. Oliveira & A.-L. Barabási Human Dynamics: The Correspondence Patterns of Darwin and Einstein. Nature 437, 1251 (2005). [PDF] Prob(RT > x) (log) Response time (log) C. Faloutsos (CMU) 67
68 T.4: duration of phonecalls Surprising Patterns for the Call Duration Distribution of Mobile Phone Users Pedro O. S. Vaz de Melo, Leman Akoglu, Christos Faloutsos, Antonio A. F. Loureiro PKDD 2010 C. Faloutsos (CMU) 68
69 Probably, power law (?)?? C. Faloutsos (CMU) 69
70 No Power Law! C. Faloutsos (CMU) 70
71 TLaC: Lazy Contractor The longer a task (phonecall) has taken, The even longer it will take Odds ratio= Casualties(<x): Survivors(>=x) == power law C. Faloutsos (CMU) 71
72 Data Description " Data from a private mobile operator of a large city " 4 months of data " 3.1 million users " more than 1 billion phone records " Over 96% of talkative users obeyed a TLAC distribution ( talkative : >30 calls) C. Faloutsos (CMU) 72
73 Outliers: C. Faloutsos (CMU) 73
74 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Belief Propagation Tensors Spike analysis Problem#3: Scalability Conclusions C. Faloutsos (CMU) 74
75 E-bay Fraud detection w/ Polo Chau & Shashank Pandit, CMU [www 07] C. Faloutsos (CMU) 75
76 E-bay Fraud detection C. Faloutsos (CMU) 76
77 E-bay Fraud detection C. Faloutsos (CMU) 77
78 E-bay Fraud detection - NetProbe C. Faloutsos (CMU) 78
79 E-bay Fraud detection - NetProbe Compatibility matrix F A H F 99% A 99% H 49% 49% heterophily C. Faloutsos (CMU) 79
80 Popular press And less desirable attention: from Belgium police ( copy of your code? ) C. Faloutsos (CMU) 80
81 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Belief Propagation Tensors Spike analysis Problem#3: Scalability Conclusions C. Faloutsos (CMU) 81
82 GigaTensor: Scaling Tensor Analysis Up By 100 Times Algorithms and Discoveries U Kang Evangelos Papalexakis Abhay Harpale Christos Faloutsos KDD 12 C. Faloutsos (CMU) 82
83 Background: Tensor Tensors (=multi-dimensional arrays) are everywhere Hyperlinks &anchor text [Kolda+,05] Anchor Text URL 2 URL Java C++ C# C. Faloutsos (CMU) 83
84 Background: Tensor Tensors (=multi-dimensional arrays) are everywhere Sensor stream (time, location, type) Predicates (subject, verb, object) in knowledge base Eric Clapton plays guitar Barrack Obama is the president of U.S. (48M) (26M) (26M) NELL (Never Ending Language Learner) data Nonzeros =144M C. Faloutsos (CMU) 84
85 Problem Definition How to decompose a billion-scale tensor? Corresponds to SVD in 2D case C. Faloutsos (CMU) 85
86 Problem Definition # Q1: Dominant concepts/topics? # Q2: Find synonyms to a given noun phrase? # (and how to scale up: data > RAM) (48M) (26M) (26M) NELL (Never Ending Language Learner) data Nonzeros =144M C. Faloutsos (CMU) 86
87 Experiments GigaTensor solves 100x larger problem GigaTensor Out of Memory 100x (K) (J) (I) Number of nonzero = I / 50 C. Faloutsos (CMU) 87
88 A1: Concept Discovery Concept Discovery in Knowledge Base C. Faloutsos (CMU) 88
89 A1: Concept Discovery C. Faloutsos (CMU) 89
90 A2: Synonym Discovery C. Faloutsos (CMU) 90
91 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Belief propagation Tensors Spike analysis Problem#3: Scalability -PEGASUS Conclusions C. Faloutsos (CMU) 91
92 Rise and fall patterns in social media Meme (# of mentions in blogs) short phrases Sourced from U.S. politics in 2008 you can put lipstick on a pig yes we can # of mentions # of mentions Time (hours) C. Faloutsos Time (CMU) (hours)
93 Rise and fall patterns in social media Can we find a unifying model, which includes these patterns? four classes on YouTube [Crane et al. 08] six classes on Meme [Yang et al. 11] C. Faloutsos (CMU)
94 Rise and fall patterns in social media Answer: YES! Value Original SpikeM Value Original SpikeM Value Original SpikeM Time Time Time 100 Original SpikeM 100 Original SpikeM 100 Original SpikeM Value 50 Value 50 Value Time Time Time We can represent all patterns by single model C. Faloutsos (CMU) In Matsubara+ SIGKDD
95 Main idea - SpikeM - 1. Un-informed bloggers (uninformed about rumor) - 2. External shock at time nb (e.g, breaking news) - 3. Infection (word-of-mouth) Time n=0 Time n=nb Time n=nb+1 β f (n) Infectiveness of a blog-post at age n: - Strength of infection (quality of news) - Decay function C. Faloutsos (CMU) 95
96 Main idea - SpikeM - 1. Un-informed bloggers (uninformed about rumor) - 2. External shock at time nb (e.g, breaking news) - 3. Infection (word-of-mouth) Time n=0 Time n=nb Time n=nb+1 Infectiveness of a blog-post at age n: β - Strength of infection (quality of news) f (n) - Decay function C. Faloutsos (CMU) f (n) = β * n
97 SpikeM - with periodicity Full equation of SpikeM % ΔB(n +1) = p(n +1) ' U(n) &' Periodicity n t=n b ( (ΔB(t)+ S(t)) f (n +1 t)+ε* )* Bloggers change their activity over time (e.g., daily, weekly, yearly) activity noon Peak 3am Dip p(n) Time n C. Faloutsos (CMU) 97
98 Details Analysis exponential rise and power-raw fall Lin-log Log-log C. Faloutsos (CMU) Rise-part SI -> exponential SpikeM -> exponential 98
99 Details Analysis exponential rise and power-raw fall Fall-part SI -> exponential SpikeM -> power law Lin-log Log-log C. Faloutsos (CMU) 99
100 Tail-part forecasts SpikeM can capture tail part C. Faloutsos (CMU) 100
101 What-if forecasting (1) First spike (2) Release date (3) Two weeks before release?? e.g., given (1) first spike, (2) release date of two sequel movies (3) access volume before the release date C. Faloutsos (CMU) 101
102 What-if forecasting SpikeM can forecast not only tail-part, but also rise-part! (1) First spike (2) Release date (3) Two weeks before release SpikeM can forecast shape of upcoming spikes C. Faloutsos (CMU) 102
103 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Belief propagation Spike analysis Tensors Problem#3: Scalability -PEGASUS Conclusions C. Faloutsos (CMU) 103
104 Scalability Google: > 450,000 processors in clusters of ~2000 processors each [Barroso, Dean, Hölzle, Web Search for a Planet: The Google Cluster Architecture IEEE Micro 2003] Yahoo: 5Pb of data [Fayyad, KDD 07] Problem: machine failures, on a daily basis How to parallelize data mining tasks, then? A: map/reduce hadoop (open-source clone) C. Faloutsos (CMU) 104
105 Roadmap Algorithms & results Centralized Hadoop/ PEGASUS Degree Distr. old old Pagerank old old Diameter/ANF old HERE Conn. Comp old HERE Triangles done HERE Visualization started C. Faloutsos (CMU) 105
106 HADI for diameter estimation Radius Plots for Mining Tera-byte Scale Graphs U Kang, Charalampos Tsourakakis, Ana Paula Appel, Christos Faloutsos, Jure Leskovec, SDM 10 Naively: diameter needs O(N**2) space and up to O(N**3) time prohibitive (N~1B) Our HADI: linear on E (~10B) Near-linear scalability wrt # machines Several optimizations -> 5x faster C. Faloutsos (CMU) 106
107 Count???? 19+ [Barabasi+] ~1999, ~1M nodes Radius C. Faloutsos (CMU) 107
108 Count?????? 19+ [Barabasi+] ~1999, ~1M nodes Radius YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Largest publicly available graph ever studied. C. Faloutsos (CMU) 108
109 Count 14 (dir.)???? ~7 (undir.) 19+? [Barabasi+] Radius YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Largest publicly available graph ever studied. C. Faloutsos (CMU) 109
110 Count 14 (dir.)???? ~7 (undir.) 19+? [Barabasi+] YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) 7 degrees of separation (!) Diameter: shrunk C. Faloutsos (CMU) Radius 110
111 Count???? ~7 (undir.) Radius YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Q: Shape? C. Faloutsos (CMU) 111
112 YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality (?!) C. Faloutsos (CMU) 112
113 Radius Plot of GCC of YahooWeb. C. Faloutsos (CMU) 113
114 YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. C. Faloutsos (CMU) 114
115 Conjecture: EN DE ~7 BR YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. C. Faloutsos (CMU) 115
116 Conjecture: ~7 YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. C. Faloutsos (CMU) 116
117 details Running time - Kronecker and Erdos-Renyi Graphs with billions edges.
118 Roadmap Algorithms & results Centralized Hadoop/ PEGASUS Degree Distr. old old Pagerank old old Diameter/ANF old HERE Conn. Comp old HERE Triangles Visualization started HERE C. Faloutsos (CMU) 118
119 Generalized Iterated Matrix Vector Multiplication (GIMV) PEGASUS: A Peta-Scale Graph Mining System - Implementation and Observations. U Kang, Charalampos E. Tsourakakis, and Christos Faloutsos. (ICDM) 2009, Miami, Florida, USA. Best Application Paper (runner-up). C. Faloutsos (CMU) 119
120 Generalized Iterated Matrix details Vector Multiplication (GIMV) PageRank proximity (RWR) Diameter Connected components (eigenvectors, Belief Prop. ) Matrix vector Multiplication (iterated) C. Faloutsos (CMU) 120
121 Example: GIM-V At Work Connected Components 4 observations: Count Size C. Faloutsos (CMU) 121
122 Example: GIM-V At Work Connected Components Count 1) 10K x larger than next Size C. Faloutsos (CMU) 122
123 Example: GIM-V At Work Connected Components Count 2) ~0.7B singleton nodes Size C. Faloutsos (CMU) 123
124 Example: GIM-V At Work Connected Components Count 3) SLOPE! Size C. Faloutsos (CMU) 124
125 Example: GIM-V At Work Connected Components Count 300-size cmpt X size cmpt Why? X 65. Why? 4) Spikes! Size C. Faloutsos (CMU) 125
126 Example: GIM-V At Work Connected Components Count suspicious financial-advice sites (not existing now) Size C. Faloutsos (CMU) 126
127 GIM-V At Work Connected Components over Time LinkedIn: 7.5M nodes and 58M edges Stable tail slope after the gelling point C. Faloutsos (CMU) 127
128 Roadmap Introduction Motivation Problem#1: Patterns in graphs Problem#2: Tools Problem#3: Scalability Conclusions C. Faloutsos (CMU) 128
129 OVERALL CONCLUSIONS low level: Several new patterns (fortification, triangle-laws, conn. components, etc) New tools: belief propagation, gigatensor, etc Scalability: PEGASUS / hadoop C. Faloutsos (CMU) 129
130 OVERALL CONCLUSIONS high level BIG DATA: Large datasets reveal patterns/ outliers that are invisible otherwise C. Faloutsos (CMU) 130
131 References Leman Akoglu, Christos Faloutsos: RTG: A Recursive Realistic Graph Generator Using Random Typing. ECML/PKDD (1) 2009: Deepayan Chakrabarti, Christos Faloutsos: Graph mining: Laws, generators, and algorithms. ACM Comput. Surv. 38(1): (2006) C. Faloutsos (CMU) 131
132 References Deepayan Chakrabarti, Yang Wang, Chenxi Wang, Jure Leskovec, Christos Faloutsos: Epidemic thresholds in real networks. ACM Trans. Inf. Syst. Secur. 10(4): (2008) Deepayan Chakrabarti, Jure Leskovec, Christos Faloutsos, Samuel Madden, Carlos Guestrin, Michalis Faloutsos: Information Survival Threshold in Sensor and P2P Networks. INFOCOM 2007: C. Faloutsos (CMU) 132
133 References Christos Faloutsos, Tamara G. Kolda, Jimeng Sun: Mining large graphs and streams using matrix and tensor tools. Tutorial, SIGMOD Conference 2007: 1174 C. Faloutsos (CMU) 133
134 References T. G. Kolda and J. Sun. Scalable Tensor Decompositions for Multi-aspect Data Mining. In: ICDM 2008, pp , December C. Faloutsos (CMU) 134
135 References Jure Leskovec, Jon Kleinberg and Christos Faloutsos Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations, KDD 2005 (Best Research paper award). Jure Leskovec, Deepayan Chakrabarti, Jon M. Kleinberg, Christos Faloutsos: Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication. PKDD 2005: C. Faloutsos (CMU) 135
136 References Jimeng Sun, Yinglian Xie, Hui Zhang, Christos Faloutsos. Less is More: Compact Matrix Decomposition for Large Sparse Graphs, SDM, Minneapolis, Minnesota, Apr Jimeng Sun, Spiros Papadimitriou, Philip S. Yu, and Christos Faloutsos, GraphScope: Parameterfree Mining of Large Time-evolving Graphs ACM SIGKDD Conference, San Jose, CA, August 2007 C. Faloutsos (CMU) 136
137 References Jimeng Sun, Dacheng Tao, Christos Faloutsos: Beyond streams and graphs: dynamic tensor analysis. KDD 2006: C. Faloutsos (CMU) 137
138 References Hanghang Tong, Christos Faloutsos, and Jia-Yu Pan, Fast Random Walk with Restart and Its Applications, ICDM 2006, Hong Kong. Hanghang Tong, Christos Faloutsos, Center-Piece Subgraphs: Problem Definition and Fast Solutions, KDD 2006, Philadelphia, PA C. Faloutsos (CMU) 138
139 References Hanghang Tong, Christos Faloutsos, Brian Gallagher, Tina Eliassi-Rad: Fast best-effort pattern matching in large attributed graphs. KDD 2007: C. Faloutsos (CMU) 139
140 Project info Thanks to: NSF IIS , IIS , CTA-INARC; Yahoo (M45), LLNL, IBM, SPRINT, C. Faloutsos (CMU) 140 Google, INTEL, HP, ilab
141 Cast Akoglu, Leman Beutel, Alex Chau, Polo Kang, U Koutra, Danae McGlohon, Mary Prakash, Aditya Papalexakis, Vagelis Tong, Hanghang C. Faloutsos (CMU) 141
142 OVERALL CONCLUSIONS high level BIG DATA: Large datasets reveal patterns/ outliers that are invisible otherwise C. Faloutsos (CMU) 142
CMU SCS Large Graph Mining Patterns, Tools and Cascade analysis
Large Graph Mining Patterns, Tools and Cascade analysis Christos Faloutsos CMU Roadmap Introduction Motivation Why big data Why (big) graphs? Patterns in graphs Tools: fraud detection on e-bay Conclusions
More informationCMU SCS Mining Large Graphs and Tensors - Patterns, Tools and Discoveries.
Mining Large Graphs and Tensors - Patterns, Tools and Discoveries. Christos Faloutsos CMU Thank you! Nikos Sidiropoulos Kuo-Chu Chang Zhi (Gerry) Tian C. Faloutsos (CMU) 2 Roadmap Introduction Motivation
More informationGraph Mining Techniques for Social Media Analysis
Graph Mining Techniques for Social Media Analysis Mary McGlohon Christos Faloutsos 1 1-1 What is graph mining? Extracting useful knowledge (patterns, outliers, etc.) from structured data that can be represented
More informationSchool of Computer Science Carnegie Mellon Graph Mining, self-similarity and power laws
Graph Mining, self-similarity and power laws Christos Faloutsos University Overview Achievements global patterns and laws (static/dynamic) generators influence propagation communities; graph partitioning
More informationScaling Up HBase, Hive, Pegasus
CSE 6242 A / CS 4803 DVA Mar 7, 2013 Scaling Up HBase, Hive, Pegasus Duen Horng (Polo) Chau Georgia Tech Some lectures are partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko,
More informationGraphs over Time Densification Laws, Shrinking Diameters and Possible Explanations
Graphs over Time Densification Laws, Shrinking Diameters and Possible Explanations Jurij Leskovec, CMU Jon Kleinberg, Cornell Christos Faloutsos, CMU 1 Introduction What can we do with graphs? What patterns
More informationBig Graph Mining: Algorithms and Discoveries
Big Graph Mining: Algorithms and Discoveries U Kang and Christos Faloutsos Carnegie Mellon University {ukang, christos}@cs.cmu.edu ABSTRACT How do we find patterns and anomalies in very large graphs with
More informationProcedia Computer Science
Procedia Computer Science 00 (2015) 1 8 Procedia Computer Science Scalable Tensor Mining Lee Sael a, Inah Jeon b, U Kang b, a Department of Computer Science, State University of New York Korea, Republic
More informationGraph Processing and Social Networks
Graph Processing and Social Networks Presented by Shu Jiayu, Yang Ji Department of Computer Science and Engineering The Hong Kong University of Science and Technology 2015/4/20 1 Outline Background Graph
More informationIntroduction to Graph Mining
Introduction to Graph Mining What is a graph? A graph G = (V,E) is a set of vertices V and a set (possibly empty) E of pairs of vertices e 1 = (v 1, v 2 ), where e 1 E and v 1, v 2 V. Edges may contain
More informationThe Internet Is Like A Jellyfish
The Internet Is Like A Jellyfish Michalis Faloutsos UC Riverside Joint work with: Leslie Tauro, Georgos Siganos (UCR) Chris Palmer(CMU) Big Picture: Modeling the Internet Topology Traffic Protocols Routing,
More informationThank you! NetMine Data mining on networks IIS -0209107 AWSOM. Outline. Proposed method. Goals
NetMine Data mining on networks IIS -0209107 Christos Faloutsos (CMU) Michalis Faloutsos (UCR) Peggy Agouris George Kollios Fillia Makedon Betty Salzberg Anthony Stefanidis Thank you! NSF-IDM 04 C. Faloutsos
More informationThe Shape of the Network. The Shape of the Internet. Why study topology? Internet topologies. Early work. More on topologies..
The Shape of the Internet Slides assembled by Jeff Chase Duke University (thanks to and ) The Shape of the Network Characterizing shape : AS-level topology: who connects to whom Router-level topology:
More informationThe ebay Graph: How Do Online Auction Users Interact?
The ebay Graph: How Do Online Auction Users Interact? Yordanos Beyene, Michalis Faloutsos University of California, Riverside {yordanos, michalis}@cs.ucr.edu Duen Horng (Polo) Chau, Christos Faloutsos
More informationSocial Network Mining
Social Network Mining Data Mining November 11, 2013 Frank Takes (ftakes@liacs.nl) LIACS, Universiteit Leiden Overview Social Network Analysis Graph Mining Online Social Networks Friendship Graph Semantics
More informationExpansion Properties of Large Social Graphs
Expansion Properties of Large Social Graphs Fragkiskos D. Malliaros 1 and Vasileios Megalooikonomou 1,2 1 Computer Engineering and Informatics Department University of Patras, 26500 Rio, Greece 2 Data
More informationCost effective Outbreak Detection in Networks
Cost effective Outbreak Detection in Networks Jure Leskovec Joint work with Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne VanBriesen, and Natalie Glance Diffusion in Social Networks One of
More informationJure Leskovec (@jure) Stanford University
Jure Leskovec (@jure) Stanford University KDD Summer School, Beijing, August 2012 8/10/2012 Jure Leskovec (@jure), KDD Summer School 2012 2 Graph: Kronecker graphs Graph Node attributes: MAG model Graph
More informationBig Data Analytics Process & Building Blocks
Big Data Analytics Process & Building Blocks Duen Horng (Polo) Chau Georgia Tech CSE 6242 A / CS 4803 DVA Jan 10, 2013 Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos
More informationTutorial, IEEE SERVICE 2014 Anchorage, Alaska
Tutorial, IEEE SERVICE 2014 Anchorage, Alaska Big Data Science: Fundamental, Techniques, and Challenges (Data Mining on Big Data) 2014. 6. 27. By Neil Y. Yen Presented by Incheon Paik University of Aizu
More informationHADI: Mining Radii of Large Graphs
HADI: Mining Radii of Large Graphs U KANG Carnegie Mellon University CHARALAMPOS E. TSOURAKAKIS Carnegie Mellon University ANA PAULA APPEL USP at São Carlos CHRISTOS FALOUTSOS Carnegie Mellon University
More informationSocial Network Analysis
Social Network Analysis Challenges in Computer Science April 1, 2014 Frank Takes (ftakes@liacs.nl) LIACS, Leiden University Overview Context Social Network Analysis Online Social Networks Friendship Graph
More informationHadoop Based Link Prediction Performance Analysis
Hadoop Based Link Prediction Performance Analysis Yuxiao Dong, Casey Robinson, Jian Xu Department of Computer Science and Engineering University of Notre Dame Notre Dame, IN 46556, USA Email: ydong1@nd.edu,
More informationLarge-scale Data Mining: MapReduce and Beyond Part 2: Algorithms. Spiros Papadimitriou, IBM Research Jimeng Sun, IBM Research Rong Yan, Facebook
Large-scale Data Mining: MapReduce and Beyond Part 2: Algorithms Spiros Papadimitriou, IBM Research Jimeng Sun, IBM Research Rong Yan, Facebook Part 2:Mining using MapReduce Mining algorithms using MapReduce
More informationSocial Networks and Social Media
Social Networks and Social Media Social Media: Many-to-Many Social Networking Content Sharing Social Media Blogs Microblogging Wiki Forum 2 Characteristics of Social Media Consumers become Producers Rich
More informationPractical Graph Mining with R. 5. Link Analysis
Practical Graph Mining with R 5. Link Analysis Outline Link Analysis Concepts Metrics for Analyzing Networks PageRank HITS Link Prediction 2 Link Analysis Concepts Link A relationship between two entities
More informationBig Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network
, pp.273-284 http://dx.doi.org/10.14257/ijdta.2015.8.5.24 Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network Gengxin Sun 1, Sheng Bin 2 and
More informationRise and Fall Patterns of Information Diffusion: Model and Implications
Rise and Fall Patterns of Information Diffusion: Model and Implications Yasuko Matsubara Kyoto University y.matsubara@ db.soc.i.kyoto-u.ac.jp Yasushi Sakurai NTT Communication Science Labs yasushi.sakurai@acm.org
More informationBig Data Analytics Building Blocks. Simple Data Storage (SQLite)
http://poloclub.gatech.edu/cse6242 CSE6242 / CX4242: Data & Visual Analytics Big Data Analytics Building Blocks. Simple Data Storage (SQLite) Duen Horng (Polo) Chau Georgia Tech Partly based on materials
More informationBig Data Analytics Building Blocks; Simple Data Storage (SQLite)
Big Data Analytics Building Blocks; Simple Data Storage (SQLite) Duen Horng (Polo) Chau Georgia Tech CSE6242 / CX4242 Jan 9, 2014 Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John
More informationBig Data Analytics Building Blocks. Simple Data Storage (SQLite)
http://poloclub.gatech.edu/cse6242 CSE6242 / CX4242: Data & Visual Analytics Big Data Analytics Building Blocks. Simple Data Storage (SQLite) Duen Horng (Polo) Chau Georgia Tech Partly based on materials
More informationPart 1: Link Analysis & Page Rank
Chapter 8: Graph Data Part 1: Link Analysis & Page Rank Based on Leskovec, Rajaraman, Ullman 214: Mining of Massive Datasets 1 Exam on the 5th of February, 216, 14. to 16. If you wish to attend, please
More informationMapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research
MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With
More informationBig Data Analytics Building Blocks; Simple Data Storage (SQLite)
Big Data Analytics Building Blocks; Simple Data Storage (SQLite) Duen Horng (Polo) Chau Georgia Tech CSE6242 / CX4242 Aug 21, 2014 Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John
More informationMMap: Fast Billion-Scale Graph Computation on a PC via Memory Mapping
: Fast Billion-Scale Graph Computation on a PC via Memory Mapping Zhiyuan Lin, Minsuk Kahng, Kaeser Md. Sabrin, Duen Horng (Polo) Chau Georgia Tech Atlanta, Georgia {zlin48, kahng, kmsabrin, polo}@gatech.edu
More informationSearch in BigData2 - When Big Text meets Big Graph 1. Introduction State of the Art on Big Data
Search in BigData 2 - When Big Text meets Big Graph Christos Giatsidis, Fragkiskos D. Malliaros, François Rousseau, Michalis Vazirgiannis Computer Science Laboratory, École Polytechnique, France {giatsidis,
More informationParallel Data Mining. Team 2 Flash Coders Team Research Investigation Presentation 2. Foundations of Parallel Computing Oct 2014
Parallel Data Mining Team 2 Flash Coders Team Research Investigation Presentation 2 Foundations of Parallel Computing Oct 2014 Agenda Overview of topic Analysis of research papers Software design Overview
More informationGuilt-by-Constellation: Fraud Detection by Suspicious Clique Memberships
Guilt-by-Constellation: Fraud Detection by Suspicious Clique Memberships Véronique Van Vlasselaer KU Leuven Veronique.VanVlasselaer @kuleuven.be Leman Akoglu Stony Brook University leman@cs.stonybrook.edu
More informationGraph Mining on Big Data System. Presented by Hefu Chai, Rui Zhang, Jian Fang
Graph Mining on Big Data System Presented by Hefu Chai, Rui Zhang, Jian Fang Outline * Overview * Approaches & Environment * Results * Observations * Notes * Conclusion Overview * What we have done? *
More informationSIAM PP 2014! MapReduce in Scientific Computing! February 19, 2014
SIAM PP 2014! MapReduce in Scientific Computing! February 19, 2014 Paul G. Constantine! Applied Math & Stats! Colorado School of Mines David F. Gleich! Computer Science! Purdue University Hans De Sterck!
More informationAnalysis of Internet Topologies: A Historical View
Analysis of Internet Topologies: A Historical View Mohamadreza Najiminaini, Laxmi Subedi, and Ljiljana Trajković Communication Networks Laboratory http://www.ensc.sfu.ca/cnl Simon Fraser University Vancouver,
More informationAnalyzing the Facebook graph?
Logistics Big Data Algorithmic Introduction Prof. Yuval Shavitt Contact: shavitt@eng.tau.ac.il Final grade: 4 6 home assignments (will try to include programing assignments as well): 2% Exam 8% Big Data
More informationAnalysis of Internet Topologies
Analysis of Internet Topologies Ljiljana Trajković ljilja@cs.sfu.ca Communication Networks Laboratory http://www.ensc.sfu.ca/cnl School of Engineering Science Simon Fraser University, Vancouver, British
More informationPresto/Blockus: Towards Scalable R Data Analysis
/Blockus: Towards Scalable R Data Analysis Andrew A. Chien University of Chicago and Argonne ational Laboratory IRIA-UIUC-AL Joint Institute Potential Collaboration ovember 19, 2012 ovember 19, 2012 Andrew
More informationGraph models for the Web and the Internet. Elias Koutsoupias University of Athens and UCLA. Crete, July 2003
Graph models for the Web and the Internet Elias Koutsoupias University of Athens and UCLA Crete, July 2003 Outline of the lecture Small world phenomenon The shape of the Web graph Searching and navigation
More informationDiscovering Social Media Experts by Integrating Social Networks and Contents
Proceedings of the Twenty-Third Australasian Database Conference (ADC 2012), Melbourne, Australia Discovering Social Media Experts by Integrating Social Networks and Contents Zhao Zhang Bin Zhao Weining
More informationGraph-Based User Behavior Modeling: From Prediction to Fraud Detection
Graph-Based User Behavior Modeling: From Prediction to Fraud Detection Alex Beutel Carnegie Mellon University abeutel@cs.cmu.edu Christos Faloutsos Carnegie Mellon University christos@cs.cmu.edu March
More informationIntroduction to Data Mining
Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:
More informationSocial Prediction in Mobile Networks: Can we infer users emotions and social ties?
Social Prediction in Mobile Networks: Can we infer users emotions and social ties? Jie Tang Tsinghua University, China 1 Collaborate with John Hopcroft, Jon Kleinberg (Cornell) Jinghai Rao (Nokia), Jimeng
More informationMining Large Graphs. ECML/PKDD 2007 tutorial. Part 2: Diffusion and cascading behavior
Mining Large Graphs ECML/PKDD 2007 tutorial Part 2: Diffusion and cascading behavior Jure Leskovec and Christos Faloutsos Machine Learning Department Joint work with: Lada Adamic, Deepay Chakrabarti, Natalie
More informationOracle Advanced Analytics 12c & SQLDEV/Oracle Data Miner 4.0 New Features
Oracle Advanced Analytics 12c & SQLDEV/Oracle Data Miner 4.0 New Features Charlie Berger, MS Eng, MBA Sr. Director Product Management, Data Mining and Advanced Analytics charlie.berger@oracle.com www.twitter.com/charliedatamine
More informationClustering Big Data. Anil K. Jain. (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012
Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012 Outline Big Data How to extract information? Data clustering
More informationBig Data: Study in Structured and Unstructured Data
Big Data: Study in Structured and Unstructured Data Motashim Rasool 1, Wasim Khan 2 mail2motashim@gmail.com, khanwasim051@gmail.com Abstract With the overlay of digital world, Information is available
More informationLarge-Scale Data Processing
Large-Scale Data Processing Eiko Yoneki eiko.yoneki@cl.cam.ac.uk http://www.cl.cam.ac.uk/~ey204 Systems Research Group University of Cambridge Computer Laboratory 2010s: Big Data Why Big Data now? Increase
More informationUnderstanding Graph Sampling Algorithms for Social Network Analysis
Understanding Graph Sampling Algorithms for Social Network Analysis Tianyi Wang, Yang Chen 2, Zengbin Zhang 3, Tianyin Xu 2 Long Jin, Pan Hui 4, Beixing Deng, Xing Li Department of Electronic Engineering,
More informationMLg. Big Data and Its Implication to Research Methodologies and Funding. Cornelia Caragea TARDIS 2014. November 7, 2014. Machine Learning Group
Big Data and Its Implication to Research Methodologies and Funding Cornelia Caragea TARDIS 2014 November 7, 2014 UNT Computer Science and Engineering Data Everywhere Lots of data is being collected and
More informationAN INTRODUCTION TO SOCIAL NETWORK DATA ANALYTICS
Chapter 1 AN INTRODUCTION TO SOCIAL NETWORK DATA ANALYTICS Charu C. Aggarwal IBM T. J. Watson Research Center Hawthorne, NY 10532 charu@us.ibm.com Abstract The advent of online social networks has been
More informationLarge Scale Social Network Analysis
Large Scale Social Network Analysis DATA ANALYTICS 2013 TUTORIAL Rui Sarmento email@ruisarmento.com João Gama jgama@fep.up.pt Outline PART I 1. Introduction & Motivation Overview & Contributions 2. Software
More informationTop 10 Algorithms in Data Mining
Top 10 Algorithms in Data Mining Xindong Wu ( 吴 信 东 ) Department of Computer Science University of Vermont, USA; 合 肥 工 业 大 学 计 算 机 与 信 息 学 院 1 Top 10 Algorithms in Data Mining by the IEEE ICDM Conference
More informationBig Graph Processing: Some Background
Big Graph Processing: Some Background Bo Wu Colorado School of Mines Part of slides from: Paul Burkhardt (National Security Agency) and Carlos Guestrin (Washington University) Mines CSCI-580, Bo Wu Graphs
More informationTop Top 10 Algorithms in Data Mining
ICDM 06 Panel on Top Top 10 Algorithms in Data Mining 1. The 3-step identification process 2. The 18 identified candidates 3. Algorithm presentations 4. Top 10 algorithms: summary 5. Open discussions ICDM
More informationExploring Big Data in Social Networks
Exploring Big Data in Social Networks virgilio@dcc.ufmg.br (meira@dcc.ufmg.br) INWEB National Science and Technology Institute for Web Federal University of Minas Gerais - UFMG May 2013 Some thoughts about
More informationBIG DATA IN BUSINESS ENVIRONMENT
Scientific Bulletin Economic Sciences, Volume 14/ Issue 1 BIG DATA IN BUSINESS ENVIRONMENT Logica BANICA 1, Alina HAGIU 2 1 Faculty of Economics, University of Pitesti, Romania olga.banica@upit.ro 2 Faculty
More informationBig Data Explained. An introduction to Big Data Science.
Big Data Explained An introduction to Big Data Science. 1 Presentation Agenda What is Big Data Why learn Big Data Who is it for How to start learning Big Data When to learn it Objective and Benefits of
More informationStudying E-mail Graphs for Intelligence Monitoring and Analysis in the Absence of Semantic Information
Studying E-mail Graphs for Intelligence Monitoring and Analysis in the Absence of Semantic Information Petros Drineas, Mukkai S. Krishnamoorthy, Michael D. Sofka Bülent Yener Department of Computer Science,
More informationAsking Hard Graph Questions. Paul Burkhardt. February 3, 2014
Beyond Watson: Predictive Analytics and Big Data U.S. National Security Agency Research Directorate - R6 Technical Report February 3, 2014 300 years before Watson there was Euler! The first (Jeopardy!)
More informationAn Analysis of Verifications in Microblogging Social Networks - Sina Weibo
An Analysis of Verifications in Microblogging Social Networks - Sina Weibo Junting Chen and James She HKUST-NIE Social Media Lab Dept. of Electronic and Computer Engineering The Hong Kong University of
More informationMapReduce Algorithms. Sergei Vassilvitskii. Saturday, August 25, 12
MapReduce Algorithms A Sense of Scale At web scales... Mail: Billions of messages per day Search: Billions of searches per day Social: Billions of relationships 2 A Sense of Scale At web scales... Mail:
More informationInfluence Propagation in Social Networks: a Data Mining Perspective
Influence Propagation in Social Networks: a Data Mining Perspective Francesco Bonchi Yahoo! Research Barcelona - Spain bonchi@yahoo-inc.com http://francescobonchi.com/ Acknowledgments Amit Goyal (University
More informationA SURVEY OF PROPERTIES AND MODELS OF ON- LINE SOCIAL NETWORKS
International Conference on Mathematical and Computational Models PSG College of Technology, Coimbatore Copyright 2009, Narosa Publishing House, New Delhi, India A SURVEY OF PROPERTIES AND MODELS OF ON-
More informationNetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks
NetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks Shashank Pandit, Duen Horng Chau, Samuel Wang, Christos Faloutsos Carnegie Mellon University Pittsburgh, PA 15213, USA
More informationAPPM4720/5720: Fast algorithms for big data. Gunnar Martinsson The University of Colorado at Boulder
APPM4720/5720: Fast algorithms for big data Gunnar Martinsson The University of Colorado at Boulder Course objectives: The purpose of this course is to teach efficient algorithms for processing very large
More informationLINEAR-ALGEBRAIC GRAPH MINING
UCSF QB3 Seminar 5/28/215 Linear Algebraic Graph Mining, sanders29@llnl.gov 1/22 LLNL-PRES-671587 New Applications of Computer Analysis to Biomedical Data Sets QB3 Seminar, UCSF Medical School, May 28
More informationW H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract
W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the
More informationBig Data Systems CS 5965/6965 FALL 2015
Big Data Systems CS 5965/6965 FALL 2015 Today General course overview Expectations from this course Q&A Introduction to Big Data Assignment #1 General Course Information Course Web Page http://www.cs.utah.edu/~hari/teaching/fall2015.html
More informationLink Prediction on Evolving Data using Tensor Factorization
Link Prediction on Evolving Data using Tensor Factorization Stephan Spiegel 1, Jan Clausen 1, Sahin Albayrak 1, and Jérôme Kunegis 2 1 DAI-Labor, Technical University Berlin Ernst-Reuter-Platz 7, 10587
More informationExample application (1) Telecommunication. Lecture 1: Data Mining Overview and Process. Example application (2) Health
Lecture 1: Data Mining Overview and Process What is data mining? Example applications Definitions Multi disciplinary Techniques Major challenges The data mining process History of data mining Data mining
More informationThe Big Data Paradigm Shift. Insight Through Automation
The Big Data Paradigm Shift Insight Through Automation Agenda The Problem Emcien s Solution: Algorithms solve data related business problems How Does the Technology Work? Case Studies 2013 Emcien, Inc.
More informationParallel Programming Map-Reduce. Needless to Say, We Need Machine Learning for Big Data
Case Study 2: Document Retrieval Parallel Programming Map-Reduce Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Carlos Guestrin January 31 st, 2013 Carlos Guestrin
More informationSunnie Chung. Cleveland State University
Sunnie Chung Cleveland State University Data Scientist Big Data Processing Data Mining 2 INTERSECT of Computer Scientists and Statisticians with Knowledge of Data Mining AND Big data Processing Skills:
More informationWeb intelligence on Big Data in Today s Life. Web intelligence on Big Data in Today s Life,
Web intelligence on Big Data in Today s Life Updesh Kumar Jaiswal I.M.S Engineering College,Ghaziabad, U.P, India updesh1984@gmail.com Abhishek Gupta I.M.S. Engineering College, Ghaziabad, U.P, India abhishekftp@yahoo.com
More informationText Analytics (Text Mining)
CSE 6242 / CX 4242 Apr 3, 2014 Text Analytics (Text Mining) LSI (uses SVD), Visualization Duen Horng (Polo) Chau Georgia Tech Some lectures are partly based on materials by Professors Guy Lebanon, Jeffrey
More informationCS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #18: Dimensionality Reduc7on
CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #18: Dimensionality Reduc7on Dimensionality Reduc=on Assump=on: Data lies on or near a low d- dimensional subspace Axes of this subspace
More informationInformation Management course
Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)
More informationBig Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012
Big Data Buzzwords From A to Z By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords Big data is one of the, well, biggest trends in IT today, and it has spawned a whole new generation
More informationBIG DATA TRENDS AND TECHNOLOGIES
BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.
More informationDynamics of information spread on networks. Kristina Lerman USC Information Sciences Institute
Dynamics of information spread on networks Kristina Lerman USC Information Sciences Institute Information spread in online social networks Diffusion of activation on a graph, where each infected (activated)
More informationDistance Degree Sequences for Network Analysis
Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation
More informationData Mining: Concepts and Techniques
Data Mining: Concepts and Techniques Chapter 1 Introduction SURESH BABU M ASST PROF IT DEPT VJIT 1 Chapter 1. Introduction Motivation: Why data mining? What is data mining? Data Mining: On what kind of
More informationMathematical Challenges in Cybersecurity
SANDIA REPORT SAND 2009-0805 Unlimited Release Printed February 2009 Mathematical Challenges in Cybersecurity Daniel M. Dunlavy, Bruce Hendrickson, and Tamara G. Kolda Prepared by Sandia National Laboratories
More informationAnalyzing Big Data with AWS
Analyzing Big Data with AWS Peter Sirota, General Manager, Amazon Elastic MapReduce @petersirota What is Big Data? Computer generated data Application server logs (web sites, games) Sensor data (weather,
More informationEvaluating Online Payment Transaction Reliability using Rules Set Technique and Graph Model
Evaluating Online Payment Transaction Reliability using Rules Set Technique and Graph Model Trung Le 1, Ba Quy Tran 2, Hanh Dang Thi My 3, Thanh Hung Ngo 4 1 GSR, Information System Lab., University of
More informationInteractive Analytical Processing in Big Data Systems,BDGS: AMay Scalable 23, 2014 Big Data1 Generat / 20
Interactive Analytical Processing in Big Data Systems,BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking,Study about DataSet May 23, 2014 Interactive Analytical Processing in Big Data Systems,BDGS:
More informationCurriculum Vitae. Summer internship in a financial company that is active in quantitative analysis or development of quantitative
Curriculum Vitae XIAOXIAO SHI Department of Computer Science University of Illinois at Chicago Office: 851 S. Morgan St., Rm 1336 SEO, Chicago, IL 60607 xshi9@uic.edu, xiao.x.shi@gmail.com (preferred)
More informationData Mining with Hadoop at TACC
Data Mining with Hadoop at TACC Weijia Xu Data Mining & Statistics Data Mining & Statistics Group Main activities Research and Development Developing new data mining and analysis solutions for practical
More informationInternational Journal of Engineering Research ISSN: 2348-4039 & Management Technology November-2015 Volume 2, Issue-6
International Journal of Engineering Research ISSN: 2348-4039 & Management Technology Email: editor@ijermt.org November-2015 Volume 2, Issue-6 www.ijermt.org Modeling Big Data Characteristics for Discovering
More informationHADI: Fast Diameter Estimation and Mining in Massive Graphs with Hadoop
HADI: Fast Diameter Estimation and Mining in Massive Graphs with Hadoop U Kang, Charalampos Tsourakakis, Ana Paula Appel, Christos Faloutsos, Jure Leskovec December 2008 CMU-ML-08-117 HADI: Fast Diameter
More informationRANKING WEB PAGES RELEVANT TO SEARCH KEYWORDS
ISBN: 978-972-8924-93-5 2009 IADIS RANKING WEB PAGES RELEVANT TO SEARCH KEYWORDS Ben Choi & Sumit Tyagi Computer Science, Louisiana Tech University, USA ABSTRACT In this paper we propose new methods for
More informationMining Large Datasets: Case of Mining Graph Data in the Cloud
Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent d Orazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining Large
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More information