Temporal Data Mining for Small and Big Data. Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant

Transcription

1 Temporal Data Mining for Small and Big Data Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant

2 What is Temporal Data Mining? Knowledge discovery in data that contain temporal information. Two types of time data: -event data (i.e., time of purchase) -time series data (EKG data). Talk Outline A. General Concepts B. Temporal Data Mining Applications: Medicine, bioinformatics, spatiotemporal data C. Temporal Data Mining and Big Data: business process, web data.

3 A. General Concepts

4 A.1 Time Data Representation and Temporality in Databases

5 Time Data Representation and Temporality in Databases Time series: Real-valued measurements at regular temporal intervals. Temporal sequences: Time stamped at regular or irregular time intervals. Example: The sequence of purchases of a customer on an online store. Transaction time: The time that information is entered in the database. For example, the time of a purchase. Valid time: The time an entity is valid in the real world. For example, the time the subscription of a customer starts. Bi-temporal time stamping. Have both a transaction and valid time.

6 Types of databases Snapshot databases: Keep the most recent version of data. Rollback databases: Support only a transaction time. Historical databases: Support only valid time. Temporal databases: Support both valid and transaction time. Allen s interval algebra Allen s interval algebra offers the most widely accepted way to express temporal relations and perform temporal reasoning [1]. Allen defines 13 temporal relations: before, after, meets, overlaps, etc.

7 Time Series Representation Requirements: Reduce the dimensionality of the similarity search problem, distance in the feature space less or equal in original space. Schemes: Fourier transform. Wavelet transform. Piecewise Aggregate Approximation and Piecewise Line Approximation. Shape Definition Language. Model-based, such as Hidden Markov model. Perceptually Important Points.

8 EKG PIP points

9 A.2. Temporal Data Mining Tasks Similarity Computation Classification/Clustering Pattern recognition Prediction

10 A.2.1 Time Series Similarity Computation

11 Similarity Computation in time series Distance-based. Dynamic Time Warping. This is applied when the time series are not aligned. Longest Common Subsequence. It assumes the same scale and baseline. It is tolerant to gaps and is more resistant to noise and outliers than DTW.

12 A.2.2 Classification/Clustering

13 Classification/Clustering of Time Series Data Non-model-based (traditional): Example: NNs, SVMs, decision trees, k-means. These can be applied to (a) features extracted from the series, such as PIPs, FT coefficients, trend, seasonality, mean, or (b) the raw time series data. Model-based. They use some model information about the time series, which comes from the fact that time series data values are usually correlated. Example: HMM, ARMA, AR, Markov chain.

14 A.2.3 Pattern Discovery in Temporal Data

15 Pattern Discovery Pattern discovery in event sequences: 1.Sequence mining (multiple sequences): Apriori, GSP.. 2. Association rule discovery (single sequence) 3. Frequent Episode Discovery (single sequence). An episode is a sequence of events appearing within a specific time window in a specific order, i.e., interest rates increase (event 1) and stock market drop (event 2). Pattern discovery in time series: 1. Motif and anomaly discovery (e.g., bioinformatics and computer networking monitoring). 2. Streaming data pattern discovery (e.g. financial data analysis or sensor data).

16 A.2.4 Prediction

17 Event prediction: Rare event prediction Prediction Event duration prediction: Regression Time Series Forecasting: Moving average Autoregression ARMA models

18 B. Applications B.1 Applications in Medicine

19 Chronus II Chronus II [3] is a temporal database mediator, that allows temporal abstractions. It extends the SQL language to allow general temporal queries on clinical databases for decisionsupport systems. On its basic level it uses Allen s interval algebra to define temporal relationships. A later ontological version [4] of the mediator exists that utilizes OWL and SWRL.

20 The TEMPADIS System This is a system for the discovery of patterns in courseof-disease data [5]. It was applied on a database of HIV patients. 18 variables were used, such as white blood cell count and drug types. Classification was performed in order to determine the health status of a patient. A decision tree approach was used. There were five health status categories ranging from asymptomatic to severe illness. Finally, the GSP algorithm was used for pattern detection in sequences of events across patients in the database.

21 Analysis and Classification of EEG Time Series In [6], the fractal dimension was used to analyze EEG signals and detect patterns. The fractal dimension was chosen because of the chaotic nature of the signals. In [7], 3 methods to classify EEG time series were compared. 1. Linear Discriminant Analysis. 2. Neural Networks. 3. Support Vector Machines. SVMs gave the best results.

22 B.2 Applications in Bioinformatics

23 General Concepts Microarray technology has enabled us to study thousands of genes simultaneously. This is done using gene expression profiles, which measure a gene s activity. Gene expression profiles can be obtained either at specific time points or at successive time intervals. In the second case, they are known as gene expression time series.

24 Clustering of Gene Expression Time Series Difficult problem because: Possible presence of noise, intersecting clusters. Time series are very short (even as short as four samples) Time series can be unevenly sampled. The time series could have different scaling and shifting. The similarity measure should be shape-based, i.e., it should be based on the changes in the intensity and not the intensity itself.

25 Clustering of Gene Expression Time Series Spline-Based Methods: can be used in time series with missing points. Model-Based Methods: For example, autoregressive equations or Hidden Markov Models can be used to model the series. Fuzzy-Clustering Based Methods Template-Based Methods: a template is used, after DTW is employed for alignment.

26 B.3 Spatiotemporal Applications

27 Analysis of moving point objects (MPOs) Two types of analysis: Descriptive modeling: Describe the entire lifeline of the moving object. Retrieval by content: Find a specific motion pattern. Descriptive Modeling Goal: Find clusters that describe the lifelines (movements). For example, the motion of a group of objects can be described using the motion azimuth. 1: the objects move in the same direction the objects move in perpendicular directions. 0: the objects move in opposite directions

28 MPO analysis: Retrieval by content Problem: Detect relative motion patterns, i.e. detect how the attributes of different object movements related over space and time (speed, change of speed, etc.) Main idea: Fit to the data a motion template with specific motion attributes Example patterns: Flocking: Objects within a circular area of radius moving in the same direction. Leadership: Objects moving in the same direction with one object being ahead of all other objects.

29 Trajectory Data Mining Problem: Find similar trajectories. This is an important problem ( e.g., object identification in video). The similarity measure must be able to handle: Different sampling rates, similar motions in different space regions, noise, data with different lengths. Approaches: LCSS [8], Minimum Bounding Rectangles [9], FT combined with SOM [10].

30 Open GeoDa Adds a Temporal Feature GeoDa [11] is a very popular open source tool for spatial analysis and modeling. In Sept. 12, it was announced that its new version will include space-time analysis maps, that will allow the user to track changes in spatial patterns over time, such as follow the change in the vegetation of an area.

31 C. Temporal Data Mining and Big Data

32 The 3 Vs of Big Data Variety Volume Velocity -> Real time/agile Analytics.

33 Agile Analytics In agile analytics, collective intelligence from the entire organization is used to develop continuously evolving prediction models as to how to enhance customer satisfaction and improve strategic business decisions.

34 C.1 Big Data and Business Processes

35 Value Chain Temporal Optimization Embedding of real-time fine granularity data in the business decision process: Real-time inventory management and efficient response to high demand times. Acquisition of real-time sensor data from the manufacturing process: - Manufacturing process efficiency: bottleneck identification, yield maximization, defect reduction. - X-raying [12] of business processes to ensure conformance with process design.

36 The Hospital and Agile Analytics Electronic medical records enable agile analytics. Possible uses: 1. Disease outbreak detection, with minimum latency. 2. Pharmacovigilance: Identification of drug adverse effects on a scale that is not possible in clinical trials.

37 C.2 Big Data from Web Usage Mining

38 Web Data Analysis for Behavioral Targeting Goals: Build behavior profiles for web users. At real-time, compute a relevance score for an ad that will decide or not the appearance of the ad. Data Mining Operations regarding users: Classification: Classify groups of users based on their profiles. Clustering: Used when the user categories are not known.

39 Mining the Web Usage Data Statistical analysis: For example, most frequently accessed web page, number of accessed web pages, maximum viewing time of a page, average length of a path to a site, etc. Path Analysis: paths. This yields the most frequent visited Association Rule Discovery: Discover the pages that are accessed together in a user session whose support exceeds a certain threshold. Sequential Pattern Discovery: Discover patterns that appear in a sequence of site visits by a user.

40 C.3 Big Data from Data Streams

41 Stream Pattern Discovery Algorithms Streaming data are of growing importance in many areas including monitoring for security purposes, financial forecasting, and analysis of location data. Challenges: 1. Huge amounts of data that arrive at high rates. 2. Often users need to respond immediately. Insight: The stream values are often correlated and a few hidden variables are enough to characterize the data.

42 Stream Pattern Discovery SPIRIT [13] : An algorithm that finds trends and hidden variables in a family of incoming streams. Main idea: Use Principle Component Analysis. Advantages : Adaptive,automatically detects changes in the incoming streams, scales linearly with the number of streams. 2. SpADe[14]: For the problem of matching an incoming stream against a predefined pattern: A warping distance that can handle shifting and scaling both in the amplitude and temporal dimensions. It can be incorporated in stream pattern discovery (in similarity search).

43 The AWSOM algorithm Purpose: For streaming data coming from sensors operating in hostile and remote environments. It allows sensors to detect patterns and trends [15]. Requirements of an algorithm that processes sensor stream data: Ability to detect simple or periodic patterns. Ability to filter out noise. Low memory usage. Be online and one pass. Ability to detect outliers. Should not require supervision by humans.

44 The AWSOM algorithm (continued) Main idea: The AWSOM algorithm utilizes wavelet primarily for the following reasons: (a) easy periodicity detection (b) need to store just a few coefficients (c) operates without supervision (d) requires only one pass. Experimental results showed that the algorithm can detect periodicities and bursts.

45 Conclusion Knowledge discovery in temporal data has applications in many areas. Since Big Data are temporal in nature, temporal data mining and especially real-time analytics and Agile Analytics are of increasing importance in order to understand the evolution of processes/customers in time and reduce the latency between data collection and using the data in decision making.

46 References 1. Allen, J. F., Maintaining Knowledge about Temporal Intervals, Communications of the ACM, vol. 26, no. 11, pp , Weiss, G.M. and H. Hirsch, Learning to Predict Rare Events in Event Sequences, Proceedings of the 4 th International Conference on Knowledge Discovery and Data Mining, pp , AAAI Press, O Connor, M.J., S.W. Tu, M.A. Musen, The Chronus II Temporal Database Mediator, Proceedings of the AMIA Annual Symposium, pp , San Antonio, TX, O Connor, M.J., R.D. Shankar, A.K.Das, An Ontology-Driven Mediator for Querying Time-Oriented Biomedical Data, 19 th IEEE International Symposium on Computer-Based Medical Systems, pp , Salt Lake City, Utah, Ramirez, J.C.G. et al., Temporal Pattern Discovery in Course-of- Disease Data, IEEE Engineering in Medicine and Biology, vol. 19, no. 4, pp , 2000.

47 References 6. Paramanathan, P. and R. Uthayakumar, Detecting Patterns in Irregular Time Series with Fractal Dimension, Proceedings of the International Conference on Computational Intelligence and Multimedia Applications, pp , [Gar03] Garrett, D. et al., Comparison of Linear, Non-Linear, and Feature Selection Methods for EEG Signal Classification, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 2, pp , June Vlachos, M., G. Kollios, D. Gunopoulos, Discovering Similar Multidimensional Trajectories, Proceedings of the International Conference on Data Engineering (ICDE), pp , Vlachos M., Hadjieleftheriou M., Gunopoulos D., Keogh E., Indexing Multi- Dimensional Time Series with Support for Multiple Distance Measures, Proceedings of the ACM SIGKDD Conference, pp , Washington DC (USA), August [Kha05] Khalid, S. and A. Naftel, Classifying Spatiotemporal Object Trajectories Using Unsupervised Learning of Basis Functions Coefficients, Proceedings of the 3 rd ACM International Workshop on Video Surveillance and Sensor Networks, pp , 2005.

48 References Van der Aalst, W., Process Mining, Communications of the ACM, pp , Papadimitriou, S., J. Sun, C. Faloutsos, Streaming Pattern Discovery in Multiple Time Series, Proceedings of the 31 st VLDB Conference, pp , Chen, Y. et al., SpADe: On Shape-Based Pattern Detection in Streaming Time Series, Proceedings of the IEEE 23 rd International Conference on Data Engineering, pp , Papadimitriou, S., A. Brockwell, C. Faloutsos, Adaptive, Unsupervised Stream Mining, The VLDB Journal, vol. 13, pp , 2004.