The Data-as-a-Service Framework for Cyber-Physical-Social Big Data

Transcription

1 The Data-as-a-Service Framework for Cyber-Physical-Social Big Data Laurence Tianruo Yang ( 杨 天 若 ) Huazhong University of Science and Technology, China St Francis Xavier University, Canada 1

2 Outline 1. Hyper (Cyber-Physical-Social)World 2. Big Data and Challenges 3. A Tensor-Based Approach 4. Illustrations and Examples 5. Conclusion Huazhong University of Science and Technology 2

3 Hyper (Cyber-Physical-Social) World Social World Physical World Cyber World Hyper World What we are and what we can do in the Hyper World? Huazhong University of Science and Technology 4

4 Hyperspace Cyber computation, communication, and control that are discrete, logical, and switched Physical natural and human-made systems governed by the laws of physics and operating in continuous time Social socially produced space, serves as a tool of thoughts, action, mind, sense, interaction, communications, etc. Hyperspace = Cyber Space + Physical Space + Social Space Huazhong University of Science and Technology 5

5 Social Space Social Comuting (Internet of People) u-human-computer Interface (BCI) Social networks Cyber networks Pervasive/Ubiquitous Intelligence Cyber Space (hybrid world) u-things (Internet of Things) u-systems/services Physical Space Physical networks 6

6 Characteristics of Hyperworld Its basic characteristic is direct mapping between virtual and real worlds via active devices including sensors, actuators, micro-machines, robots, etc. -- Kunii, Ma & Huang, Keynote Hyperworld Modeling in VIS 96 C Data/Info Explosion! Last 5 years web data > 5,000 years whole data C C C Connection Explosion! C2C D2D H2H M2M CPS T2T (IoT) Service/App Explosion! WbS SaaS PaaS IaaS EaaS Cloud Intelligence Explosion! Ubi-Sensing CA TTT AmI Smart W/P Huazhong University of Science and Technology 7

7 Intelligent Computing Waves 1 st : AI (Logic/KL-based, cyber-based) - Machine learning - NLP & Comp-Vision - Robot & game theory - Expert system - Knowledge/Reasoning - DAI (Distributed AI) 3 rd : Social-based - Autonomous software - Multi/Massive agents - Agent language - Agent negotiation & cooperation - Personal/social behavior - Web intelligence/semantics 2 nd : Physical-based - Probabilistic computing - Fuzzy logic - Neural network - GA/Evolutionary computing - Chaotic/Swarm computing - Biologic computing 4 th : Cyber-Physical-Social - Physical/everyday things intelligence - Atop of the above 3 intelligent comp - Scale, dynamic, heterogeneous, spontaneous - Predictable, controllable, adaptable, manageable, ethic, - Others-aware & self-aware mind/spirit? Huazhong University of Science and Technology 8

8 HyperWorld Individuals Companies/Societies Data Social Space Social networks Internet /WWW SEA-net Data Cyber space Cyber networks Data Physical Space Physical networks Huazhong University of Science and Technology 10

9 Workstation Workstation Agents Workstation Workstation Workstation Workstation Social World Workstation Human Companies/Societies Socio-culture & organizational components Individuals activities Services Wisdom Knowledge Information Data Data Cycle Cyber World Things The Internet of Things Physical World Huazhong University of Science and Technology 11

10 Outline 1. Smart Word and Hyper World 2. Big Data and Challenges 3. A Tensor-Based Approach 4. Illustrations and Examples 5. Conclusion Huazhong University of Science and Technology 12

11 Big Data What s the difference between large data and big data? Large Data: GB Massive Data: TB Big Data: PB Just Size? Huazhong University of Science and Technology 13

12 Features of Big Data 4Vs Features Volume Terabytes, Petabytes, Volume Variety Structured Semi-Structured Un-Structured Velocity Variety Big Data Veracity Streaming Data Processing time requirement Veracity Velocity Quality of Data Huazhong University of Science and Technology 14

13 Cyber-Physical-Social Data Huazhong University of Science and Technology 15

14 Challenges Caused by Big Data Volume Large Scale Data, Limited Compute Resources Variety Diversified Format, Unified Representation Velocity Requirement for quick processing Veracity Inconsistent, Incomplete, Redundant Data Huazhong University of Science and Technology 16

15 How to tackle the problems caused by the 4Vs of Big Data? Huazhong University of Science and Technology 17

16 Outline 1. Smart Word and Hyper World 2. Big Data and Challenges 3. A Tensor-Based Framework 4. Illustrations and Examples 5. Conclusion Huazhong University of Science and Technology 18

17 A Tensor-Based Approach 3.1 Tensor and Tensor Decomposition 3.2 4Vs Problem with Tensor 3.3 A Framework for Big Data Analysis and Mining Huazhong University of Science and Technology 19

18 Tensor Definition Mathematics: Tensor Computer Science: Multiple Dimensional Arrays One Order Tensor: Vector Three Order Tensor Two Order Tensor: Matrix L.Ddecomposition. SIAM Journal of Matrix Analysis and Applications, 21(4), pages , (2000) Huazhong University of Science and Technology 20

19 Tensor Computations and Algebra Kronecker product. Tensor-tensor product Inverse Identity tensor Transpose Tensor-matrix mode product Stefan Ragnarsson, Ph.D thesis, 2012, Cornell Equivalently: Unfolding on mode 1 (take mode 1 fibers and linearize).

20 Illustration Model-N Unfolding Huazhong University of Science and Technology 22

21 Illustration d 2 d 1 d 3 d 2 d 2 d 1 d 3 unfold transpose d 2 d 1 d 3 transpose d 2 d 1 d 3 fold d 1 d 2 d 3 Rotate 90º clockwise. d 1 d 3 d 1 Rotate and repeat. d 3 d 2 d 3 d 2 d 1 d 2 d 1 d 3

22 Formal Description a, i I, j I, k I, l I, m I, n I ijklmn t x y z c u Huazhong University of Science and Technology 24

23 Tensor Decomposition Tensor decomposition for dimension reduction Big data are modeled as tensors Big Data Huazhong University of Science and Technology 25

24 Huazhong University of Science and Technology 26

25 Decomposition Process Data SVD Matrix data SVD Approximation Data A (1) A (2) a a 11 1n m1 a a mn A U V Coresets A (k) Huazhong University of Science and Technology 27

26 Huazhong University of Science and Technology 28

27 Singular Value Decomposition (SVD) X = U V T X U V T 1 2 v 1 x (1) x (2) x (M) = u 1 u 2 u k.. v 2 k v k singular values right singular vectors input data left singular vectors 2007 Jimeng Sun

28 Huazhong University of Science and Technology 30

29 Huazhong University of Science and Technology 31

30 Huazhong University of Science and Technology 32

31 Huazhong University of Science and Technology 33

32 Huazhong University of Science and Technology 34

33 Huazhong University of Science and Technology 35

34 Huazhong University of Science and Technology 36

35 Huazhong University of Science and Technology 37

36 Huazhong University of Science and Technology 38

37 Huazhong University of Science and Technology 39

38 Huazhong University of Science and Technology 40

39 Huazhong University of Science and Technology 41

40 Huazhong University of Science and Technology 42

41 Huazhong University of Science and Technology 43

42 Huazhong University of Science and Technology 44

43 Incremental SVD A U V 0 Update the singular vectors and singular values by using the incremental decomposition results of matrix A1 Incremental Matrix 1 ' ' ' V 0 A0 A U J U V 0 I Updated U Updated V Updated Embedded and Pervasive Computing Lab 45

44 HO-SVD Tensor Decomposition Approximation Assemble Model-N Unfolding Huazhong University of Science and Technology 46

45 Approximate Tensor Calculation Matrix Unfolding SVD for Every Unfolding Matrix c Approximate Tensor Core Tensor Huazhong University of Science and Technology 47

46 Incremental HOSVD Original SVD Incremental SVD Huazhong University of Science and Technology 48

47 A Tensor-Based Approach 3.1 Tensor and Tensor Decomposition 3.2 4Vs Problem with Tensor 3.3 A Framework for Big Data Analysis and Mining Huazhong University of Science and Technology 49

48 Variety Volume <University> <Student> <Category> One frame <ID> <Name> <Research> Variety Big Data Veracity I y I x I name I name I y I x I id I id I t I id Velocity Huazhong University of Science and Technology 50

49 Frame y a Video z l k j x XML Documents RDF OWL m n i t Users Social Relationships c 6-Order Tensor u

50 Variety Embedded and Pervasive Computing Lab 52

51 I s GPS Video I u L. Kuang, F. Hao, L.T. Yang, M. Lin, C. Luo and G. Min, A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction, IEEE Transactions on Emerging Topics in Computing (TETC), 2014, DOI: /TETC I t XML Document Unified Representation for Big Data L. Kuang, F. Hao, L.T. Yang, M. Lin, C. Luo and G. Min, A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction, published at IEEE Transactions on Emerging Topics in Computing (TETC), 2014.

52 Velocity Recalculation Volume time Variety Big Data Veracity t n t 3 t 2 t 1 Velocity CPU Memory... CPU Memory Huazhong University of Science and Technology 54

53 研 究 内 容 3: 大 数 据 简 约 计 算 方 法 研 究 正 交 基 增 量 张 量 增 量 核 心 张 量 增 量 Updated Basis Incremental Data 55

54 Incremental HO-SVD Huazhong University of Science and Technology 56

55 Veracity Volume Variety Big Data Veracity Original Tensor Core Tensor Velocity Extract High Quality Core Tensor from PB(TB) Level Medial Tensor Error Constraints Huazhong University of Science and Technology 57

56 Volume Volume High Order Tensor Variety Big Data Value Matrix Unfolding A A( u ) () t... Velocity Incremental Lanczos at matrix level Ci 1 C i k Ci 1 C i k Three Layers Incremental Parallel Strategies Two layers incremental SVD parallel strategies on servers Huazhong University of Science and Technology 58

57 Transparent Computing vs. Cloud Computing Computation in sever: Synthetic SVD Results Compute Core Tensor All computation are performed in sever SVD Incremental SVD Compute Core Tensor Transparent Server Transparent Network Cloud Computing Transparent Client Computation in client: SVD Incremental SVD Embedded and Pervasive Computing Lab 59

58 Distributed Computing with Multi-Objective Optimization f 31 f 32 f f 21 f 22 f... TN f 11 f 12 f Computation Time Memory Usage Energy Consumption Communication Traffic OPT : minz Tim Mem Egy Tra L. Kuang, Y. Zhang, L.T. Yang, J. Chen, F. Hao, and C. Luo, A Holistic Approach to Distributed Dimensionality Reduction of Big Data, Accepted at IEEE Transactions on Cloud Computing (TCC), Huazhong University of Science and Technology 60

59 A Tensor-Based Approach 3.1 Tensor and Tensor Decomposition 3.2 4Vs Problem with Tensor 3.3 A Framework for Big Data Analysis and Mining Huazhong University of Science and Technology 61

60 Big Data Framework Huazhong University of Science and Technology 62

61 The Framework 1. Secure Dimensionality Reduction 2. Deep Computation Models 3. Big Data Ranking and Retrieval Huazhong University of Science and Technology 63

62 1. Secure Dimensionality Reduction Huazhong University of Science and Technology

63 1. Secure Dimensionality Reduction Huazhong University of Science and Technology 65

64 1. Secure Dimensionality Reduction L. Kuang, L.T. Yang, and J. Feng, A Novel Secure Big Data Dimensionality Reduction Scheme, Accepted at IEEE Transactions on Cloud Computing (TCC), Huazhong University of Science and Technology 66

65 2. Big Data + Tensor +Deep Learning = Deep Computation Basic Deep Computation Model: Basic Deep Computation Model: Input/Output: Tensor-Based Data A General Deep Learning Model for Big Data Hidden: Tensor-Based Features Unsupervised Feature Learning Parameter: Tensor-Based Parameters Bridge Between Representation and Mining Clustering Classification Prediction Deep Computation Model Tensor-Based Representation Huazhong University of Science and Technology

66 2. Big Data + Tensor+ Deep Learning = Deep Computation Basic Deep Computation Model: Basic Deep Computation Model Input/Output: Tensor-Based Data Tensor-Based Computing Hidden: Tensor-Based Features Tensor Distance Data Distribution Parameter: Tensor-Based Parameters Cost Function Tensor Distance Huazhong University of Science and Technology I 1 I 2 I N X R x X i I j N l i 12 i... i N j j N ; (1, 1 ) j 1 l i ( i 1) I 1 j 2 j I 1 I 2 I N T TD l, m 1 l m l l m m d g ( x y ) ( x y ) ( x y ) G( x y ) g lm 2 1 p p 2 2 exp{ l m 2 } 2 2 p p ( i i ) ( i i ) ( i i ) l m N N Q. Zhang, L.T. Yang, and Z. Chen, A Deep Computational Model for Unsupervised Feature Learning for Big Data, IEEE Transactions on Service Computers (TSC), major revision, 2015.

67 2. Incremental Deep Computation Model Incremental Model Requirements: Key Challenges: Adaption: Learn features on new samples Parameters updating incrementally: nonlinear optimization problem Preservation: Preserve previous results Structure updating incrementally: adapting the structure of tensor autoencoder with preserving the previous knowledge Fast: Update parameters and structure in real-time Huazhong University of Science and Technology

68 2. Incremental Deep Computation Model Parameters Updating Incrementally: Structure Updating Incrementally: Update: parameters to Update: Structure and Parameters Preserving: Structure Idea: Incorporate neutrons into hidden layers Objective function: Special case: auto-encoder Approximation: 1 ( x, ) T T ( x, ) 1 T J ( x; ) (( ) x) (( ) x) 2 2 Solution: General case: Q. Zhang, L.T. Yang, and Z. Chen, An Incremental Deep Computation Model for Big Data Feature Learning, IEEE Transactions on Service Computers (TSC), Major Revision, Huazhong University of Science and Technology

69 2. Secure Deep Computation Model on Cloud Goals: Overall Scheme: Improve efficiency: Using cloud computing Protect privacy: Adopting BGV encryption Challenges: Choose high efficient full encryption scheme Activation function approximation Client: Only encryption and decryption operations Cloud: All the computing tasks Q. Zhang, L.T. Yang, and Z. Chen, Privacy Preserving Deep Computation Model on Cloud for Big Data Feature Learning, Accepted at IEEE Transactions on Computers (TC), Huazhong University of Science and Technology

70 3. Big Data Ranking and Retrieval Video 5-order Tensor XML Documents 4-order Tensor Users Tensorization 3-order Tensor Data Space: Represent heterogeneous data as sub tensors. Huazhong University of Science and Technology 72

71 3. Big Data Ranking and Retrieval Sub-Tensor 子 张 量 Sub-Tensor Sub-Tensor... (Relation 关 联 张 量 Tensor) 子 张 量 Sub-Tensor Sub-Tensor Metric Space: Map all the sub tensors to a Hamming Space, and obtain the similarity between all sub tensors. The similarities are stored in the relation tensor. Huazhong University of Science and Technology 73

72 Huazhong University of Science and Technology 74

73 Huazhong University of Science and Technology 75

74 3. Big Data Ranking and Retrieval Sub-Tensor r = T n r Vector r denotes the rank of the data object Huazhong University of Science and Technology 76

75 3. Big Data Ranking and Retrieval Similarity Tensor Distance... I1 I2... I N d g ( x y )( x y ) TD lm l l m m lm, T ( x y) G( x y) In the metric space, the similarity is computed with tensor distance. g lm 1 P P l m 2 exp L. Kuang, L.T. Yang, and H. Liu, 3T5R: A Tensor-based Data-as-a-Service Framework for Cyber-Physical-Social Big Data, Accepted at IEEE Transactions on Cloud Computing (TCC), Huazhong University of Science and Technology 77

76 Outline 1. Smart Word and Hyper World 2. Big Data and Challenges 3. A Tensor-Based Approach 4. Illustration and Example 5. Conclusion Huazhong University of Science and Technology 81

77 Illustration 1: Huazhong University of Science and Technology 82

78 研 究 内 容 4: 校 园 大 数 据 环 境 下 人 的 影 子 数 据 应 用 实 例 子 张 量 子 张 量 子 张 量 Original Tensor Shadow Data 学 生 Relation Tensor 子 张 量 Sub Tensor Sub Tensor Shadow Data 学 生 学 生 子 张 量 子 张 量 子 张 量 子 张 量 子 张 量 Cloud V 2 Client 正 交 基 增 量 Tensor Deep Computation for Mining and Analysis Basis : T V 1 S SubTensor 2 V 3 n n 核 心 张... Original 量 增 量 Core Tensor Tensor 张 量 增 量 OPT minz F F F 83

79 Illustration 2: Huazhong University of Science and Technology 84

80 Heterogeneous Data from Intelligent Device Huazhong University of Science and Technology 85

81 Four Step to Construct Six-Order Tensor Smart Home Social Space: Father, Mother, Children Physical Space: Bedroom, Dining room Cyber Space: Computer, Electronic devices Huazhong University of Science and Technology 86

82 Initial Tensor Huazhong University of Science and Technology 87

83 Tensor Decomposition Huazhong University of Science and Technology 88

84 Approximate Tensor Construction Huazhong University of Science and Technology 89

85 Likelihood Generated by Approximation Tensor Huazhong University of Science and Technology 90

86 Outline 1. Smart Word and Hyper World 2. Big Data and Challenges 3. A Tensor-Based Approach 4. Illustration and Example 5. Conclusion Huazhong University of Science and Technology 91

87 Problem A tensor-based framework for big data Representations, Relations, Reductions, Retrieval, Reasoning and Recommendations! Huazhong University of Science and Technology 92