AMiner (ArnetMiner) Toward understanding big scholar data. Jie Tang Tsinghua University

Transcription

1 AMiner (ArnetMiner) Toward understanding big scholar data Jie Tang Tsinghua University [1] J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD 08. pp

2 An author-centric academic search and mining system

3 Expert Search Finding experts, for data mining 3 Demographics: gender, language, location, etc. Similar Authors Knowledge about data mining

4 Researcher Profile Basic Info. Research Interests Citation Statistics 4 Ego Network

5 p Academic search and mining system p Online since 006 p p p p >136 million researcher profiles >101 million publication papers >341 million requests AMiner 100K IP access from 170 countries per month p Deep analysis, mining, and search 5

6 User Distribution 8.3 million IP from 0 countries/regions 6

7 User Distribution 8.3 million IP from 0 countries/regions Top 10 countries 1. USA 6. Canada. China 7. Japan 3. Germany 8. Spain 4. India 9. France 5. UK 10. Italy 7

8 Technologies How to populate a semantic-based profile database for researchers? Extraction Integration Mining 8

9 1 1 9 Ruud Bolle DBLP: Ruud Bolle EE EE EE EE EE Extracting Semantics from the Web (ACM TKDD, WWW 1, ISWC 06, ICDM 07, ACL 07) Office: 1S-D58 Letters: IBM T.J. Watson Research Center video database indexing Contact Information Ruud Bolle P.O. Box 704 video processing 1 Office: 1S-D58 Yorktown Heights, NY USA visual human-computer interaction Letters: Packages: IBM T.J. Watson IBM T.J. Research Watson Center Research Center IBM T.J. Watson biometrics applications P.O. Box Skyline Drive Research Center Hawthorne, NY 1053 USA IBM T.J. Watson Research Yorktown Heights, NY USA Packages: IBM bolle@us.ibm.com T.J. Watson Research Center Research_Interest Center Research Staff 19 Skyline Drive 19 Skyline Drive Ruud M. Bolle was born in Voorburg, The Netherlands. He received the Bachelor's Affiliation Hawthorne, NY 1053 USA Hawthorne, NY 1053 USA Degree in Analog Electronics in bolle@us.ibm.com 1977 and the Master's Degree in Electrical Address IBM T.J. Watson Research Engineering in 1980, both from Delft University of Technology, Delft, The ecvg/people/bolle.html Position Center Ruud M. Netherlands. Bolle was born In 1983 in Voorburg, he received The the Netherlands. Master's Educational Degree He received in Applied the History Bachelor's Mathematics and in P.O. Box 704 Degree in 1984 Analog the Ph.D. Electronics in Electrical in 1977 Engineering and the Master's from Brown Degree University, in Electrical Providence, Rhode Address Yorktown Heights, Island. In 1984 he became a Research Staff Member at the IBM Thomas J. Watson Homepage Engineering in 1980, both from Delft University of Technology, Delft, The NY USA Netherlands. Research In 1983 Center in the Artificial Intelligence Department of Computer Science Department. Two he received the In 1988 he became questions: Master's Degree in Applied Mathematics and in 1984 the Ph.D. in Electrical Engineering from manager Brown of University, the newly formed Providence, Exploratory Rhode Computer Ruud Bolle Photo bolle@us.ibm.com Island. In Vision 1984 Group he became which a is Research part of the Staff Math Member Sciences at the Department. Name IBM Thomas J. Watson Research Center in the Artificial Intelligence Department of the Computer Science 1984 Phddate Ruud Bolle Bsdate 1977 Department. Currently, In 1988 his he research became interests manager are of focused the newly on formed video database Exploratory indexing, Computer video Phduniv Bsuniv Vision Group processing, which visual part human-computer of the Math Sciences interaction Department. and biometrics applications. Brown University 1 Phdmajor Delft University of Technology How to accurately extract Electrical Engineering the semantic-based Bsmajor Currently, Ruud his research M. Bolle interests is a Fellow are focused the IEEE on and video the database AIPR. He indexing, is Area Editor video of Computer Msdate processing, Vision visual and human-computer Image Understanding interaction and Associate and biometrics Editor applications. of Pattern Recognition. Ruud Analog Electronics 1980 M. Bolle is a Member profile the IBM Academy information of Academic Technology. Services from the Web? Msuniv Msmajor Msmajor Ruud M. Bolle is a Fellow of the IEEE and the AIPR. He is Area Editor of Computer Delft University of Technology Electrical Engineering Vision and Image Understanding and Associate Editor of Pattern Recognition. Ruud Co-author Applied Mathematics Co-author M. Bolle is a Member of the IBM Academy of Technology. How to deal with the name ambiguity problem? 006 Nalini K. Ratha, Jonathan Connell, Ruud M. Bolle, Sharat Chikkerur: Cancelable Biometrics: A Case Study in Fingerprints. ICPR (4) 006: Sharat Chikkerur, Sharath Pankanti, Alan Jea, Nalini K. Ratha, Ruud M. Bolle: Fingerprint Representation Using Localized Texture Features. ICPR (4) 006: Andrew Senior, Arun Hampapur, Ying-li Tian, Lisa Brown, Sharath Pankanti, Ruud M. Bolle: Appearance models for occlusion handling. Image Vision Comput. 4(11): (006) 005 Ruud M. Bolle, Jonathan H. Connell, Sharath Pankanti, Nalini K. Ratha, Andrew W. Senior: The Relation between the ROC Curve and the CMC. AutoID 005: 15-0 Sharat Chikkerur, Venu Govindaraju, Sharath Pankanti, Ruud M. Bolle, Nalini K. Ratha: Novel Approaches for Minutiae Verification in Fingerprint Images. WACV. 005: Publications Title Cancelable Biometrics: A Case Study in Fingerprints Publication #3 Publication #5 Publication 1# ICPR 370 Date Venue End_page Start_page coauthor coauthor Title Fingerprint Representation Using Localized Texture Features Ruud Bolle Publication # ICPR 51 Date Venue End_page Start_page affiliation position UIUC Professor

10 Researcher Network Extraction [1,] Research_Interest Affiliation Postion Person Photo Homepage Name Phddate Phduniv Phdmajor Coauthor Researcher Msdate Msuniv Fax Phone Address Msmajor Authored Bsdate Bsuniv Bsmajor Title Publication_venue Start_page Publication End_page Date 70.60% of the researchers have at least one homepage or an introducing page 85.6% from universities 71.9% are homepages 40% are in lists and tables 14.4% from companies 8.1% are introducing pages 60% are natural language text There are a large number of person names having the ambiguity problem Even 3 Yi Li graduated the author s lab 70% moved at least one time [1] J. Tang, L. Yao, D. Zhang, and J. Zhang. A Combination Approach to Web User Profiling. ACM Transactions on Knowledge Discovery from Data (TKDD), (vol. 5 no. 1), Article (December 010), 44 pages. 10 [] J. Tang, D. Zhang, and L. Yao. Social Network Extraction of Academic Researchers. ICDM 07. pp

11 Our Approach Picture based on Markov Random Field Markov Property: Y a Y b { } PY ( Y Y Y ) i j j i { } = PY ( Y Y ~ Y ) i j j i Y d Y c Y e Special Cases: - Conditional Random Fields - Hidden Markov Random Fields Y f y 4= y 9=3 y 1=1 co-conference y =1 cite y 3=1 t-coauthor y 7= cite y 5= y 6= cite co-conference y 8=1 coauthor y 10=3 coauthor coauthor y 11=3 coauthor x 4 x 9 11 Researcher Profiling x 1 Name Disambiguation x x 3 x 5 x 6 x 7 x 11 x 8 x 10

12 Researcher Profile Database Extracted more than 1,000,000 researcher profiles from the Web J. Tang, L. Yao, D. Zhang, and J. Zhang. A Combination Approach to Web User Profiling. ACM Transactions on Knowledge Discovery from Data (TKDD), (vol. 5 no. 1), Article (December 010), 44 pages. 1

13 Name Disambiguation Name Affiliation Shanghai Jiao Tong Univ. Yunnan Univ. Tsinghua Univ. Jing Zhang (6) Alabama Univ. Univ. of California, Davis Carnegie Mellon University Henan Institute of Education - How to perform the assignment automatically? - How to estimate the person number? [1] J. Tang, A.C.M. Fong, B. Wang, and J. Zhang. A Unified Probabilistic Framework for Name Disambiguation in Digital Library. IEEE Transaction on Knowledge and Data Engineering (TKDE), Volume 4, Issue 6, 01, Pages [] X. Wang, J. Tang, H. Cheng, and P. S. Yu. ADANA: Active Name Disambiguation. ICDM 11, pages , 011.

14 Our Method to Name Disambiguation y 1 =1 co-conference y =1 cite y 3 =1 y 4 = y 9 =3 t-coauthor y 7 = coauthor cite y 5 = y 10 =3 y 6 = coauthor cite coauthor co-conference y 11 =3 y 8 =1 coauthor A hidden Markov Random Field model Observable Variables X represent publications x 4 x 9 Hidden Variables Y represent the labels of publications x 1 x x 3 x 5 x 6 x 7 x 8 x 11 x 10 Paper relationships define the dependencies over hidden variables J. Tang, A.C.M. Fong, B. Wang, and J. Zhang. A Unified Probabilistic Framework for Name Disambiguation in Digital Library. IEEE Transaction on Knowledge and Data Engineering (TKDE), Volume 4, Issue 6, 01, Pages

15 Disambiguation Performance ADANA Sacluser CONSTRAINT HAC DISTINCT

16 Still problems? P1: error propagation in the profile extraction Homepage finding Profile extraction from the identified pages P: the accurately extracted semantics are not really accurate? 16

17 17 Is this Enough?

18 Semantics are distributed Wikipedia Homepage LinkedIn AMiner 18

19 Integrating Semantics across Networks Identifying users from multiple heterogeneous networks and integrating semantics from the different networks together. LinkedIn WikiPedia Jeannette Wing Jeannette Wing Google Scholar 19 AMiner

20 Local + Global consistency AMiner v 1 1 Network matching 网络一致性 v 1 v 1 G 1 v 1 1 Username: Ortiz_Brandy Nation: USA Gender: female v 3 1 Global 全局一致性 inconsistency Local 局部一致性 consistency v 1 3 Nation: USA Gender: female v 1 v 1 x 1 v 1 1 v 1 v 1 3 x v 1 3 x Inconsistent! G v 1 v 1 3 v v 3 v 3 v 3 3 G 3 x v 1 1 x v 1 x v 1 3 0

21 COSNET: Connecting Social Networks with Local and Global Consistency f e (y 1, y ) y f e (y, y 3 ) G 1 v 1 1 y 1 f e (y, y 4 ) f e (y, y 5 ) y 3 v 1 1 v 1 v 1 1 v v 1 1 v 3 v 1 1 v 1 v 1 1 v 3 y 4 y 5 v 1 v 3 1 s v 1 v 1 v 1 v v 1 v 3 v 1 v f l (x 1 ) f l (x ) f l (x 4 ) f l (x 5 ) f l (x 3 ) G v v 1 v 3 Matching Graph Generation v 3 1 v 1 v 3 1 v v 3 1 v 3 Candidate Pruning v 3 1 v 1 v 3 1 v 3 Model Construction x 1 x x 3 v 1 1 v 1 x 4 x 5 v 3 1 v 1 v 1 v v 3 1 v 3 v 1 1 v 3 (a) Two input networks (b) The generated matching graph (c) Matching graph after pruning (d) The constructed model Yutao Zhang, Jie Tang, Zhilin Yang, Jian Pei, and Philip Yu. COSNET: Connecting Heterogeneous Social Networks with Local and Global 1 Consistency. KDD 15, page

22 AMiner

23 AMiner Ego Network 3

24 Network Semantics How to mine semantics within networks? Who are Jiawei Han s students and advisor? 4

25 Mining Network Semantics Input: Temporal collaboration network 1999 Output: Semantic Relationship network analysis (0.9, [/, 1998]) Ada 000 Bob Ada (0.4, [/, 1998]) (0.5, [/, 000]) 000 (0.8, [1999,000]) Ying Smith Jerry Bob (0.7, [000, 001]) (0.65, [00, 004]) Ying (0.49, [/, 1999]) Jerry (0., [001, 003]) 004 Smith C. Wang, J. Han, Y. Jia, J. Tang, D. Zhang, Y. Yu, and J. Guo. Mining Advisor-Advisee Relationships from Research Publication 5 Networks. KDD'10, pages 03-1.

26 Partially Labeled Pairwise Factor Graph Model (PLP-FGM) Constraint factor h Input: Social Network v 4 v 3 v 5 v v 1 PLP-FGM y 1 =Friend y 1 =advisor y 1 Latent Variable h (y 1, y 1 ) y 1 g (y 1, y 34 ) g (y 1,y 45 ) f(x,x 1,y 1 ) f(x 1,x,y 1 ) y 1 =advisee y 1 =Friend r 1 r 34 r 1 y 34 =? y 34 f(x 3,x 4,y 34 ) g (y 45, y 34 ) y 45 y 34 y 16 =coauthor y 16 =Other f(x 4,x 5,y 45 ) r 34 r 45 Wenbin Tang, Honglei Zhuang, and Jie Tang. Learning to Infer Social Ties in Large Networks. In ECML/PKDD'011. pp (Best Student Paper Runner-up) y 34 =? f(x 3,x 4,y 34 ) Correlation factor g relationships Attribute factors f Input Model Map relationship to nodes in model Example: Example: #Coauthored Author papers A has by a two longer authors? publication history than both B and C.

27 AMiner Ego Network 7

28 Why you will love AMiner? finding collaborations, applying PhD, reviewing, recruiting Related publications: ACM TKDD (), IEEE TKDE (3), KDD 08-15, WWW 1-15,J. Informetrics, SIGMOD 09, IJCAI 09-15, ICML 14 8

29 Examples Expert search When starting a Work on a new research topic; Or brainstorming for novel ideas. Researcher Who are experts in this field? What are the top conferences in the field? What are the best papers? What are the top research labs? 9

30 Examples Collaboration Recommendation Data Mining Theory Large Graph Heterogeneous Network Social Network 30

31 Examples More Challenging Questions Organization or Agency Looking for researchers from the U.S. who can speak Chinese Want to recruit female researchers working on WSDM 31

32 Examples Reviewer Suggestion Conf. Committee Journal Editors Who are best matching reviewers for each paper? Who will accept the invitation to review the paper Paper content 3

33 33 Applications

34 Data Mining Experts 34

35 35 NSFC (NSF of China) proposal reviewer recommendation

36 Integration with ScholarOne Find the extension reviewer recommender from Chrome Web Store 36 ScholarOne is a submission management system used by many journals such as IEEE Transactions and ACM Transactions.

37 37 Knowledge Trend for AI

38 Organization By different metrics Ranking 38

39 Widely used.. l l The largest publisher: Elsevier Conferences KDD WSDM 011 ICDM 011 ICDM 01 SocInfo 011 ICMLA 011 WAIM 011 etc. 39

40 40 Academic Social Platform

41 41

42 4

43 43

44 44

45 45 Open Big Scholar Data

46 Citation Network 46

47 Collaboration Network with Rich Semantics 47

48 AMiner (ArnetMiner) Toward understanding big scholar data Semantics + Social + Open Data Science of Science! [1] J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD 08. pp

49 AMiner (ArnetMiner) Toward understanding big scholar data Thanks to our partners [1] J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD 08. pp

50 AMiner (ArnetMiner) Toward understanding big scholar data Special 50 thanks to Yuxiao Dong from ND for helping cook the slides!

51 Why AMiner.org Academic search was treated as document search, but ignored semantics 51