Keeping Pace with Big Data

Transcription

1 - A Data Mining Perspec>ve Huan Liu, Tempe, AZ hep:// NSF Workshop on Big Data Analy6cs for Infrastructure and Building Resilience and Sustainability, Beijing, China Sept 19-20, 2014 NSF Workshop on Big Data Analy6cs, Beijing 1

2 Concluding Remarks Big Data is a good problem to have Data mining is one way of approaching it Together, we can harness it for beler sci & eng Arizona State University Data Mining and Machine Learning Lab Keeping Pace with Big Data NSF Workshop on Big Data Analy6cs, Beijing 2 2

3 Big data is not a new problem, but a persistent one Why now? We re overwhelmed, start apprecia6ng data value, and data is generated ubiquitously (we re part of the problem) We have been dealing with it since we had data Feature selec6on, as an example, to balle data explosion (mainly for alribute- value data) Big data will only become bigger Ubiquitous and fast growing linked data in the age of social media Example con6nued, Feature selec6on for linked data Big data is a good problem to have And, many a 6me, big data may not be big enough NSF Workshop on Big Data Analy6cs, Beijing 3

4 Data will only become bigger hlp://iot.ieee.org/newsleler/september- 2014/the- internet- of- things- the- story- so- far.html NSF Workshop on Big Data Analy6cs, Beijing 4

5 Begin with AEribute- Value Data It is the most familiar form of data we encounter Tables in Excel, Databases, Data is conveniently collected everywhere Some typical challenges Data overload (increasing in both width and length) Data is collected for various reasons Data accumulates at an unprecedented speed Data itself does not offer any insight, but has poten6al To make sense of massive amounts of data is to focus: using only relevant data Data preprocessing is an important part of machine learning and data mining Feature selec6on is an effec6ve approach to downsizing data NSF Workshop on Big Data Analy6cs, Beijing 5

6 Massive Data and High Dimensionality Dimensionality of data has increased exponen6ally 10,000,000 log Max # Features of UCI data set 1,000, ,000 # Features 10,000 1, s 1990s 2000s max 102 1,558 3,231,961 NSF Workshop on Big Data Analy6cs, Beijing 6

7 A General Model of KDD Knowledge Discovery and Data Mining Data mining Applying analy6cal methods and tools to discover ac6onable palerns, construct sta6s6cal or predic6ve models, and iden6fy rela6onships among massive data NSF Workshop on Big Data Analy6cs, Beijing 7

8 Why Feature Selec>on? Most machine learning and data mining techniques may not be effec6ve for high- dimensional data Curse of Dimensionality Query accuracy and efficiency degrade rapidly as the dimensionality increases. The intrinsic dimensionality may be small. For example, the number of genes responsible for a certain type of disease may be small. NSF Workshop on Big Data Analy6cs, Beijing 8

9 Classifica>on A process of predic6ng the classes of unseen instances based on palerns learned from available instances Supervised learning with labeled data Classifica>on Rules Training Data Classifica>on Algorithm If Hair = blonde and Loca>on = no, then sunburned Test Data New Data NSF Workshop on Big Data Analy6cs, Beijing 9

10 Clustering A process of grouping objects (or instances) into clusters so that objects are similar to one another within a cluster but dissimilar to objects in other clusters Unsupervised learning with unlabeled data Clustering tasks NSF Workshop on Big Data Analy6cs, Beijing 10

11 Applica>ons of Feature Selec>on Customer rela6onship management Text mining and visual analy6cs Image retrieval Microarray data analysis and protein classifica6on Face recogni6on and handwrilen digit recogni6on Intrusion detec6on Social media and social networking apps NSF Workshop on Big Data Analy6cs, Beijing 11

12 Online Document Classifica>on Web Pages The image cannot be displayed. Your The image cannot be displayed. computer Your may computer not have enough memory may not have enough memory to open to open the image, the or the image may image, or the image may have been corrupted. Restart your Restart your computer, and computer, then open and the then file open the file again. again. If the red x still appears, If the red you x may still have appears, to you may have delete the image and then to insert delete it the again. image and then insert it again. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. s Documents D 1 D 2 D M Terms T 1 T 2. T N C Sports Travel Jobs Internet ACM Portal IEEE Xplore Digital Libraries PubMed n n n Task: To classify unlabeled documents into categories Challenge: thousands of terms Solu>on: to apply dimensionality reduc6on NSF Workshop on Big Data Analy6cs, Beijing 12

13 Gene Expression Microarray Analysis Expression Microarray Image Courtesy of Affymetrix Task: To classify novel samples into known disease types (disease diagnosis) Challenge: hundreds of thousands of genes, but a few samples Solu>on: Feature Selec6on Expression Microarray Data Set NSF Workshop on Big Data Analy6cs, Beijing 13

14 Other Types of High- Dimensional Data Face images HandwriLen digits NSF Workshop on Big Data Analy6cs, Beijing 14

15 Evalua>on Measures for Ranking and Selec>ng Features The goodness of a feature/feature subset is dependent on measures Various measures Informa6on measures Distance measures Dependence measures Consistency measures Accuracy measures NSF Workshop on Big Data Analy6cs, Beijing 15

16 Informa>on Measures Entropy of variable X Entropy of X aher observing Y Informa6on Gain NSF Workshop on Big Data Analy6cs, Beijing 16

17 How to Validate Selec>on Results Direct evalua6on (if we know a priori ) Ohen suitable for ar6ficial data sets Based on prior knowledge about data Indirect evalua6on (if we don t know ) Ohen suitable for real- world data sets Based on a) number of features selected, b) performance on selected features (e.g., predic6ve accuracy, goodness of resul6ng clusters), and c) speed NSF Workshop on Big Data Analy6cs, Beijing 17

18 Methods for Result Evalua>on Accuracy For one ranked list Learning curves For results in the form Number of Features of a ranked list of features Before- and- aher comparison For results in the form of a minimum subset Comparison using different classifiers To avoid learning bias of a par6cular classifier Repea6ng experimental results For non- determinis6c results NSF Workshop on Big Data Analy6cs, Beijing 18

19 A Recent Book for Further Informa>on Six Chapters 1. Data of High Dimensionality and Challenges 2. Univariate Formula6on of Spectral Feature Selec6on (SFS) 3. Mul6variate Formula6ons 4. Connec6ons to Exis6ng Algorithms 5. Large- Scale SFS 6. Mul6- Source SFS Algorithms with sohware are available at dmml.asu.edu/sfs NSF Workshop on Big Data Analy6cs, Beijing 19

20 From ALribute- Value Data to Linked Data - We are living in an increasingly connected world NSF Workshop on Big Data Analy6cs, Beijing 20

21 Tradi>onal Media and Data Broadcast Media One- to- Many Communica6on Media One- to- One Tradi>onal Data NSF Workshop on Big Data Analy6cs, Beijing 21

22 Linked Data in the Age of Social Media Social Networking Content Sharing Social Media Blogs Wikis Forums NSF Workshop on Big Data Analy6cs, Beijing 22

23 Social Media: Many- to- Many Everyone can be a media outlet or producer Disappearing communica6on barrier Dis6nct characteris6cs User generated content: Massive, dynamic, extensive, instant, and noisy Rich user interac6ons: Linked data Collabora6ve environment: Wisdom of the crowd Many small groups: The long tail phenomenon; and ALen6on is hard to get NSF Workshop on Big Data Analy6cs, Beijing 23

24 Noise Removal Fallacy in Social Media We ohen learn that: Noise should be removed before data mining; and 99% TwiLer data is useless. Had eggs, sunny- side- up, this morning Can we remove noise as we usually do in DM? What is leh aher noise removal? TwiLer data can be rendered useless aher conven6onal noise removal As we are certain there is noise in data and there is a peril of removing it, what can we do? NSF Workshop on Big Data Analy6cs, Beijing 24

25 Linked Data and AEribute- Value Data They exist for different purposes Rela6ons, Connec6ons, or Links Proper6es, Content, etc. Classic machine learning and data mining methods assume independent, iden6cally distributed or i.i.d. property for alribute- value data Addi6onal challenges with the confluence of alribute- value and linked data User- generated Large Noisy, short, incomplete Unstructured, or free form NSF Workshop on Big Data Analy6cs, Beijing 25

26 Feature Selec>on for Social Media Data Massive and high- dimensional social media data poses unique challenges to data mining tasks Scalability Curse of dimensionality Social media data is inherently linked A key difference between social media data and alribute- value data Jiliang Tang and Huan Liu. ``Feature Selec6on with Linked Data in Social Media'', SIAM Interna6onal Conference on Data Mining (SDM), NSF Workshop on Big Data Analy6cs, Beijing 26

27 Feature Selec>on of Social Media Data Feature selec6on has been widely used to prepare large- scale, high- dimensional data for effec6ve data mining Tradi6onal feature selec6on algorithms deal with only flat" data (a2ribute- value data). Independent and Iden6cally Distributed (i.i.d.) We need to take advantage of linked data for feature selec6on NSF Workshop on Big Data Analy6cs, Beijing 27

28 Representa>on for Social Media Data u 1 p 1 p 2 f m... c k. u 1 u 2 u 3 u 4 u 2 u 3 u 4 p 4 p 5 p 6 p 7 p ser- post rela6ons NSF Workshop on Big Data Analy6cs, Beijing 28

29 Representa>on for Social Media Data u 1 p 1 p 2... f m c k. u 1 u 2 u 3 u 4 u 2 u 3 u 4 p 4 p 5 p 6 p 7 p User- user rela6ons NSF Workshop on Big Data Analy6cs, Beijing 29

30 Representa>on for Social Media Data u 1 p 1 p 2... f m c k. u 1 u 2 u 3 u 4 u 2 u 3 u 4 p 4 p 5 p 6 p 7 p Social Context NSF Workshop on Big Data Analy6cs, Beijing 30

31 Problem Statement Given labeled data X and its label indicator matrix Y, the dataset F, its social context including user- user following rela6onships S and user- post rela6onships P, Select k most relevant features from m features on dataset F with its social context S and P NSF Workshop on Big Data Analy6cs, Beijing 31

32 How to Use Link Informa>on The new ques6on is how to proceed with addi6onal informa6on for feature selec6on Two basic technical problems Rela6on extrac6on: What are dis6nc6ve rela6ons that can be extracted from linked data Mathema6cal representa6on: How to use these rela6ons in feature selec6on formula6on Do we have theories to guide us in this effort? NSF Workshop on Big Data Analy6cs, Beijing 32

33 Rela>on Extrac>on u 4 p 8 u 1 u 3 p 7 p 6 p 1 p 2 p 3 u 2 p 4 p 5 1. CoPost 2. CoFollowing 3. CoFollowed 4. Following NSF Workshop on Big Data Analy6cs, Beijing 33

34 Rela>ons, Social Theories, Hypotheses Social correla6on theories suggest that the four rela6ons may affect the rela6onships between posts Social correla6on theories Homophily: People with similar interests are more likely to be linked Influence: People who are linked are more likely to have similar interests Thus, four rela6ons lead to four hypotheses for verifica6on NSF Workshop on Big Data Analy6cs, Beijing 34

35 NSF Workshop on Big Data Analy6cs, Beijing 35 Modeling CoFollowing Rela>on Two co- following users have similar topics of interests ) ( ^ k F f i T k F f i k F f W F f T u T k i k i = )= ( Users' topic interests + + u N u u j i F T u j i u T u T, 2 2 ^ ^ 2,1 2 W ) ( ) ( W Y W X min β α

36 Evalua>on Results on Digg NSF Workshop on Big Data Analy6cs, Beijing 36

37 Evalua>on Results on Digg NSF Workshop on Big Data Analy6cs, Beijing 37

38 Summary LinkedFS is evaluated under varied circumstances to understand how it works. Link informa6on can help feature selec;on for social media data. Unlabeled data is more ohen in social media, unsupervised learning is more sensible, but also more challenging. Jiliang Tang and Huan Liu. `` Unsupervised Feature Selec6on for Linked Social Media Data'', the Eighteenth ACM SIGKDD Interna6onal Conference on Knowledge Discovery and Data Mining, Jiliang Tang, Huan Liu. ``Feature Selec6on with Linked Data in Social Media'', SIAM Interna6onal Conference on Data Mining, NSF Workshop on Big Data Analy6cs, Beijing 38

39 Looking Ahead New, rich data sources like social media present challenges and opportuni6es Feature selec6on is shown here for illustra6on Challenges abound Data collec6on (sampling bias, is data enough?) Data prepara6on (what is noise?) PaLern discovery (content, context, networks) Evalua6on (when without ground truth) Big data allows more opportuni6es for researchers of different disciplines to conduct collabora6ve research NSF Workshop on Big Data Analy6cs, Beijing 39

40 Thank You For this opportunity to share our research Acknowledgments Grants from NSF, ONR, and ARO, among others DMML members and project leaders Collaborators NSF Workshop on Big Data Analy6cs, Beijing 40

41 Concluding Remarks Big Data is a good problem to have Data mining is one way of approaching it Together, we can harness it for beler sci & eng Arizona State University Data Mining and Machine Learning Lab Keeping Pace with Big Data 41 NSF Workshop on Big Data Analy6cs, Beijing 41