Big Data Analytics: Challenges and What Computational Intelligence

Transcription

1 Big Data Analytics: Challenges and What Computational Intelligence Techniques May Offer Ah-Hwee Tan ( School of Computer Engineering Nanyang Technological University Big Data Analytics Symposium Big Data Analytics Symposium London, UK 13 September 2013

2 Outline Big Data Analytics Computational Intelligence Techniques Web Data Analytics Flexible Organizer for Competitive Intelligence (FOCI) Web Information Fusion and Associative Discovery Analytics for Active Living for Elderly

3 The Era of Big Data Big data refers to collection of data sets so large and complex that t exceed dthe competence of commonly used IT systems in terms of processing space and/or time.

4 Sources of Big Data Traditionally, mostly produced in scientific fields such as astronomy, meteorology, genomics physics, biology, and environmental research. With rapid development of IT technology and the consequent decrease of cost on collecting and storing data, big data has been generated from almost every industry and sector as well as governmental department, including retail, finance, banking, security, audit, electric power, healthcare. Recently, big data over the Web (big Web data for short), which includes all the context data, such as, user generated contents, browser/search log data, deep web data, etc.

5 Examples of Big Data (Source: Wikipedia) Walmart handles more than 1 million customer transactions every hour, which h is imported into databases estimated t to contain more than 2.5 petabytes (2560 terabytes) of data the equivalent of 167 times the information contained in all the books in the US Library of Congress. Facebook handles 50 billion photos from its user base. FICO Falcon Credit Card Fraud Detection System protects 2.1 billion active accounts world-wide. Windermere Real Estate uses anonymous GPS signals from nearly 100 million drivers to help new home buyers determine their typical drive times to and from work throughout various times of the day.

6 Examples of Big Data (Source: Wikipedia) NASA Center for Climate Simulation (NCCS) stores 32 petabytes of climate observations and simulations on the Discover supercomputing cluster. Utah Data Center is a data center currently being constructed cted by the United States National Security Agency. When finished, the facility will handle yottabytes of information collected by NSA over the Internet. Value Metric 1000 kb kilobyte MB megabyte GB gigabyte TB terabyte PB petabyte EB exabyte ZB zettabyte YB yottabyte

7 Money of Big Data (Source: Wikipedia) "Big data" have increased the demand of information management specialists Software AG, Oracle Corporation, IBM, Microsoft, SAP, EMC, and HP have spent more than $15 billion on software firms specializing in data management and analytics. In 2010, this industry on its own was worth more than $100 billion and was growing at almost 10 percent a year: about twice as fast as the software business as a whole.

8 Market of Big Data (Source: Wikipedia) Developed economies make increasing use of data- intensive technologies. There are 4.6 billion mobilephone subscriptions worldwide and there are between 1 billion and 2 billion people accessing the internet The world's effective capacity to exchange information through telecommunication networks was 281 petabytes in 1986, 471 petabytes in 1993, 2.2 exabytes in 2000, 65 exabytes in 2007 [14] and it is predicted that the amount of traffic flowing over the internet will reach 667 exabytes annually by [5]

9 Big Data Market Segments (Report by Transparency Market Research) Segmentation of the big data market by components, by applications and by geography. The different components included are software and services, hardware and storage. Software and services segment dominates the components market whereas storage segment will be the fastest growing segment for the next 5 years owing to the perpetual growth in the data generated.

10 Big Data Market Segment by Applications Covered eight applications namely financial services, manufacturing, healthcare, telecommunication, government, retail and media & entertainment and others in the application segment. Financial Services, healthcare and the government sector are the top three contributors of the big data market and together held more than 55% of the big data market in Media and Entertainment t t and the healthcare sectors will grow at high CAGR of nearly 42% from 2012 to The growth in data in the form of video, images, and games is driving the media and entertainment segment. Read more:

11 Challenges of Big Data Volume Size in the order of petabytes, exabytes, Velocity Time sensitive data, data that grow exponentially or even in rates that overwhelm the wellknown Moore's Law Variety Value Metric 1000 kb kilobyte MB megabyte GB gigabyte TB terabyte PB petabyte EB exabyte ZB zettabyte YB yottabyte From structured data into semi-structured and completely unstructured data of different types, such as text, image, audio, video, click streams, log files,

12 Deeper Issues of Big Data (The additional 3Vs) Validity Is the data correct and accurate for the intended usage? Veracity Are the results meaningful for the given problem space? Volatility How long do you need to look/store this data?

13 Computational Intelligence Neural Networks (IJCNN) Brain-like mathematical models for pattern recognition, memory, and association discovery Examples: Perceptron, BP, SVM, SOM, ART, Fuzzy Systems (IEEE-FUZZ) Fuzzy operators for handling non-discrete reasoning Examples: FNN, Fuzzy C-Means,

14 Computational Intelligence Evolutionary Computing (CEC) Classes of heuristic algorithms repeatedly search for good solutions by mimicking the process of natural evolution Commonly used for optimization and search problems Examples: Genetic Algo, Memetic Algo,

15 Flagship Events of Computational Intelligence World Congress on Computational Intelligence (Australia 2012, Beijing 2014) IEEE Symposium on Computational Intelligence (Singapore 2013, Florida, USA 2014) IEEE Symposium on Computational Intelligence in Big Data (IEEE CIBD'2014)

16 Examples of Use of CI in Big Data Data size and feature space adaptation Uncertainty modeling in learning from big data Distributed learning techniques in uncertain environment Uncertainty in cloud computing Distributed ib parallel l computation Feature selection/extraction in big data Sample selection based on uncertainty Incremental Learning Manifold Learning on big data Uncertainty techniques in big data classification/clustering Imbalance learning on big data Active learning on big data Random weight networks on big data Transfer learning on big data

17 Self-Organizing i Neural Networks for Personalized Web Intelligence Towards Personalized Web Intelligence g Ah-Hwee Tan, Hwee-Leng Ong, Hong Pan, Jamie Ng, Qiu-Xiang Li Knowledge and Information Systems 18 (2004)

18 Workflow for Web Data Analytics Search Getting the information Organize (clustering/categorizing) Putting things in perspectives Analyze (data mining) Discover hidden knowledge Share (knowledge management) Saving for reference and sharing Track Constant monitoring

19 Approaches to Organizing/Analyzing Clustering Organizing information into groups based on similarity functions and thresholds e.g. BullsEye, NorthernLight, Vivisimo Categorization Organizing information into a predefined set of classes e.g. Yahoo!, Autonomy Knowledge Server Which is better?

20 Clustering Pros Unsupervised/self-organizing, require no training or predefinition of classes Able to identify new themes Cons Users have no control Ever changing cluster structure Difficult to navigate and track

21 Categorization Pros Good control on classes Every info assigned to one or more classes of interests Cons Require learning (supervised) and/or definition of classification rules/knowledge Every info has to be assigned to one or more classes Good control but lack flexibility to handle new information

22 User-configurable Clustering (Tan & Pan, PAKDD 2002) Information organization ation and content management Online incremental clustering + userdefined structure (preferences) Reduces to a clustering system if no user indication given Allows personalization in a direct, intuitive, and interactive manner Control + flexibility

23 ARAM for Personalized Information Management Information Clusters F 2 F 1 a F 1 b a a - x x b - b + + Information Vector A B Preference Vector

24 Flexible Organizer for Competitive Intelligence (FOCI) A platform for gathering, organizing, tracking, analyzing, and sharing competitive information Natural way of turning raw search results into personalized CI portfolios Multilingual enabled with Multilingual Efficient Analyzer Domain localization (Technology) Patented and licensed to many companies

25 FOCI User Interface

26 FOCI Architecture Intranet/ Internet User s CI Portfolio Domain-Specific Knowledge Content Gathering Content Management Content Publishing Content Analysis Front End Visu ualization

27 Personalized Content Management Portfolio created through Search Unsupervised clustering (ARAM Pattern Channel A) Loop Personalization by users (ARAM Pattern Channel B) Reorganization of clusters (ARAM Pattern Channel A&B) Saving of personalized portfolio Tracking of new information

28 Personalization Functions Marking/labeling (selected) clusters Personal interpretation Inserting Clusters Indicate preference on groupings Merging clusters Indicate preferences on similarities Splitting clusters... Indicate preferences on differences

29 Information Clustering A portfolio created by a meta-search of 4 search engines with a query on Text Mining

30 A Personalized Portfolio after <=19 personalization operations (mainly labeling and creating clusters)

31 Organizing g New Information Without the Personalized Portfolio Based on Personalized Portfolio 42 new documents from DirectHit, Netscape, and BusinessWire

32 Summary A fusion neural network algorithm, called fusion ART, has been proposed for integrating clustering and categorization Has been applied to competitivee intelligence on the web. Comparing with existing works, fusion ART has advantages in Personalization fusion ART performs analysis and organization of data based on user preferences Low time complexity fusion ART performs real-time search and match of patterns resulting in a linear time complexity Incremental clustering manner fusion ART may adapt to dynamic web multimedia li data set by incrementally clustering new patterns based on the learnt cluster structure without referring to the old data. 3 2

33 Heterogeneous Data Co-clustering for Social Media Data Theme Discovery and Mining Lei Meng, Ah-Hwee Tan and Dong Xu IEEE Transactions on Knowledge and Data Engineering,

34 Introduction The popularity p of social websites leads to greatly increase of web multimedia documents Massive number Billions of images and articles online Diversity Diverse content and booming emerging topics Multi-modal descriptors images, text, category, tags, comments Category Birds Images Wild, bird, beach, tree, vacation, animal, mar, sunny, playa, nayarit, arena,ave, water, vacaciones, hollyday, pelicano. Keywords from Surrounding text 34

35 Introduction Clustering of web multimedia data is challenging Scalability to big data Difficulty in integrating multi-modal feature data Ambiguity in deciding the number of categories Rich but noisy meta-information semantic gap of images, noisy tags Birds Beach Wild, bird, beach, tree, vacation, animal, mar, sunny, playa, nayarit, arena, ave, water, vacaciones, hollyday, pelicano. Ocean, blue, sea, summer, vacation, sun, man, beach, water, yellow, fun, sand, play, funny, adult, humor, lifestyle, sunny, resort. 35

36 Problem Statement We define the theme discovery of web multimedia data as a heterogeneous data co-clustering problem, which h identifies the semantic categories of data patterns through the fusion and recognition of multiple types of features. Apple Multiple Descriptions Category Tag Apple Fruits Products Movies User Description Surrounding text 36

37 Proposed Approach A self-organizing neural network approach to Heterogeneous Data Co-clustering Based on Fusion Adaptive Resonance Theory (Fusion ART) Fuse arbitrary number of feature modalities Adaptively tune the weights for different feature modalities Two different learning function for primary data, such as images and articles, and meta-information to handle short and noisy text t Incremental fast learning Do not need to give the number of clusters 37

38 Experiments NUS-WIDE data set images of 18 categories Visual features: Grid color moment, Edge direction histogram, and wavelet texture Textual features of surrounding text: t 1142 words (7 words per image on average) 20 Newsgroups data set text documents of 10 categories Textual features of document content: over 60k words (800 words per document on average) Textual features of category: 3 labels per document on average 38

39 Experiments on NUS-WIDE Data Set Evaluation on weight adaptation across channels for visual and textual features Performance Comparison with fixed weight values GHF-ART with the adaptively tuned weight values γ_sa achieves the best performance in 5 classes and the overall performance, and achieves close performance with the best results obtained by fixed weight values 39

40 Experiments on NUS-WIDE Data Set Tracking of the change in weight values of γ _SA Textual features of surrounding text are assigned higher weights than visual features The value of γ_sa stabilizes in [0.7, 0.8] with the increase of patterns Big fluctuation may be resulted by the generation of new clusters 40

41 Experiments on NUS-WIDE Data Set Clustering Performance comparison with existing algorithms in terms of weighted average precision, cluster entropy ( H l H cluster), class entropy ( class ), purity and rand index (RI) GHF-ART achieves the best performance in terms of all the evaluation measures With supervisory information, GHF-ART(SS) consistently obtains better performance 41

42 Experiments on NUS-WIDE Data Set Time complexity analysis GHF-ART and Fusion ART incur very small increase of time cost For images, GHF-ART complete the clustering process in 10 seconds 42

43 Experiments on 20 Newsgroups Data Set Clustering performance comparison using document content and category information Both GHF-ART and GHF-ART(SS) outperform other algorithms in all the evaluation measures GHF-ART has a 5% gain than Fusion ART in terms of Average Precision, Purity and Rand Index. Comparing with other unsupervised algorithms, GHF-ART achieves around 80% in Average Precision, Purity and Rand Index while other algorithms typically obtain less than 75% 43

44 Summary A Heterogeneous data co-clustering algorithm, called GHF- ART, is proposed to discover the themes of web multimedia data via their rich but heterogeneous descriptors. Comparing with existing works, GHF-ART has advantages in Strong noise immunity A learning function of meta-information is proposed to handle noise Adaptive channel weighting ihti A well-defined dfi dweighting i algorithm is proposed to identify the important feature modalities for a better fusion of multi-modal features for overall similarity measure; Low time complexity GHF-ART performs real-time search and match of patterns resulting in a linear time complexity for big data; Incremental clustering manner GHF-ART may adapt to dynamic web multimedia data set by incrementally clustering new patterns based on the learnt cluster structure without referring to the old data. 44

45 Research Centre of Excellence in Ati Active LIving for the elderly (LILY) Aging in Place: Opportunities and Challenges Ah-Hwee Tan ( School of Computer Engineering Nanyang Technological University JOINT UBC-NTU RESEARCH CENTRE

46 Aging g in Place the ability to live in one's own home and community safely, independently, and comfortably, regardless of age, income, or ability level - Center for Disease Control, Dec

47 Motivation Global aging population creates silver challenges Most adults would prefer to age in place 78 percent of adults between the ages of 50 and 64 report that they would prefer to stay in their current residence as they age Growing elderly population will be living independently in own homes Vital to transform future homes into intelligent human-centered environment for the elderly Golden opportunities for innovating assistive technologies for aging in place 47

48 A Basic Scenario of Tender Care for Agingin-place Unobtrusive Sensing Social Signal Processing Context Aware Auto Tagging Social Cognitive Network Unobtrusive sensing device detects: the elder keeps walking around at an irregular pace. Social signal processing indicates: the elder has been silent for an unusually long time. Cognitive Analysis result Your mother may be feeling anxious now I need to call my mother now

49 Silver Challenges 49

50 Vision To enable elderly l to maintain i an active, healthy and engaging life style in their own homes supported by an age-friendly intelligent environment, providing all- round comprehensive tender care Round-the-clock day-to-day health and wellness monitoring i Cognitive Support and recommendation to products and services Companionship and emotional support Support for maintaining/stimulating social interaction 50

51 Design Consideration and Challenges How to perform unobtrusive monitoring? - Mobile sensing, activity tracking How to provide all-around around comprehensive care? - Physical, cognitive, emotional, social, sustainability How to maintain ubiquitous access and interaction? - Cross platform, multimedia, multimodal How to provide friendly, personal touch? - Adaptive user modeling, mood detection - Proactive, natural interactioni 51

52 Approach and Methodology To support active living of elderlies through an intelligent multi-agent environment with ubiquitous access, natural interface, and alldd comprehensive rounded care Key Technologies Unobtrusive sensing and social signal processing Activity pattern and user modeling Information and service recommendation Proactive stimulation and natural interaction 52

53 A Multi-Agent Collaborative Care Environment Isabel (Personal Nurse) Alfred (The Butler) Small talk User modeling Social and travel advisory Frank (Robot Dog) Activity sensing Pattern modeling Small talk Recommendations for healthcare products and services 53

54 Why Multi-Agent? Unobtrusive sensing and monitoring agents of different characteristics and capabilities Ubiquitous i access to information i and services agents in different platforms and locations Comprehensive tender care agents with different domain knowledge and functions Three s a party more opportunities for cognitive stimulation and social interaction 54

55 Comprehensive Tender Care Physical Support Activity tracking, safety and wellness monitoring Cognitive i Support information i and recommendation on (healthcare) products, services, skills and activities t Emotional Support mood detection, affective support, small talk Social Support companionship and connection to family and friends (old and new) through sms, s and facebooks etc 55

56 Unobtrusive Sensing and Ubiquitous Access to Services unobtrusive in-home real-time data collection and contextual social signal processing - Essential to better understand and cater to the elderly s l needs. Sensing bio sensing, motion sensors, wearable/mobile sensors for health monitoring and activity tracking Cross Platform Large screen interactive display, mobile handheld devices, physical robots Multimedia text, audio, video 56

57 Adaptive User Modelling Identity and profile Interests and preferences Behaviour model: Time, space, activity it Knowledge and skills Social network: Familyand friends Meth0ds for Model Building Explicit: User specification Implicit: User actions, choices, conversation 57

58 Cognitive Support: Product/Service Recommendation Domain knowledge: Healthcare, Travel, Cooking Delivery modes: - Question & Answer - Proactive recommendation - Conversation PersonalTouch: Personalized, Context sensitive, small talks 58

59 Challenges in Big Living Analytics Volume huge amount of data through bio sensing, motion sensors, wearable/mobile sensors for health monitoring and activity tracking Velocity 24x7 real time sensing, sense making, decision making, service recommendation Variety information integration and knowledge sharing from cross platform, multimedia unstructured data - text, audio, video, gestures 59

60 Research Centre of Excellence in Active LIving for the elderly (LILY) Thank you! JOINT UBC-NTU RESEARCH CENTRE