# Foundations - 2. Periodicity Detection, Time-series Correlation, Burst Detection. Temporal Information Retrieval

Save this PDF as:

Size: px
Start display at page:

Download "Foundations - 2. Periodicity Detection, Time-series Correlation, Burst Detection. Temporal Information Retrieval"

## Transcription

1 Foundations - 2 Periodicity Detection, Time-series Correlation, Burst Detection Temporal Information Retrieval

2 Time Series An ordered sequence of values (data points) of variables at equally spaced time intervals

3 Periodicity Detection How does one identify periodic values

4 Periodicity Detection Time-series is in the time domain Method1 (DFT): Identify the underlying periodic patterns by transforming into the frequency domain Method 2 (Autocorrelation) Correlate the signal with itself Find dominant frequencies

5 Fourier Transform A signal has an amplitude (strength), frequency (periodicity) and phase (offset) phase frequency amplitude Fourier Transform converts a signal from the time domain to the frequency domain

6 Discrete Fourier Transform (DFT) A Fourier analysis is a method for expressing a function as a sum of periodic components, and for recovering the function from those components. When both the function and its Fourier transform are replaced with discretized counterparts, it is called the discrete Fourier transform (DFT). fourier transform f 4 f 3 f 2 f 1 f 0 inv. fourier transform Advantages of DFT apart from periodicity detection? denoising, compression

7 Discrete Fourier Transform (DFT) fourier coefficients fourier transform F[n] periodogram f (Hz) sinusoid X(f k/n )= p 1 NX 1 N n=0 x(n) e j2 kn N The fourier coefficients encode both the amplitude and phase

8 Power Spectral Density (PSD) Estimation fourier transform F[n] f (Hz) To find out the dominant frequency we need to find the power at each frequency Periodogram encodes the strength at a given frequency P(f k/n )= X(f k/n ) 2 k =0, 1... N 1 2

9 PSD estimation using Periodogram time series data or signal P2= P1= 7 periodogram P(f k/n )= X(f k/n ) 2 k =0, 1... N To find the dominant frequencies choose the top-k dominant frequencies

10 Disadvantages of the Periodogram time series data or signal P2= P1= 7 periodogram P(f k/n )= X(f k/n ) 2 k =0, 1... N Good only for short and medium periodicities Spectral leakage - frequencies not integral multiples of the DFT bin spread over other bins false alarms

11 Autocorrelation Correlate the time series with itself Peaks get amplified ACF ( ) = P N i=1 Y i Y i+ N Fine-grained periodicity detector

12 Autocorrelation time series data or signal 0 x day Auto-correlation ACF ( ) = P N i=1 Y i Y i+ N To determine dominant period significance threshold needs to be specified Multiples of the same period are also peaks needs post processing

13 Auto-Period Auto-correlation : Good for large periods but difficult to automatically determine periods Periodogram : Easy to threshold but not accurate for short periods Idea: Get candidate periods from Periodogram and validate false alarms using Auto-correlation hill Sequence Periodogram Candidate Autocorrelation Refine Period Period Periods valley False Alarm Dismiss

14 Auto-Period P1= P2= P3= Periodogram Circular Autocorrelation Valley: False Alarms Hill: Candidate Period 357: correct period

15 Matching Time Series Similar time series suggest similar things

16 Matching Time Series Correlating time series used for clustering, classification, anomaly detection, speech recognition etc.

17 Matching Time Series What measure would you use to match two time series? d = X t y t x t Euclidean Distance Why is Euclidean matching not good enough?

18 Dynamic Time Warping Time series might be shifted Time series might be compressed at some point in time Random noise at some points Dynamic time warping measures the distance between two sequences under certain restrictions. Not a metric. Triangle inequality doesn't hold

19 Detour - Edit Distance Edit distance measures how many steps it takes to convert a string to another based on restrictions Restrictions define cost function insertion, deletion, replacement f o x f o x f f a a x x insertions and deletions insertions, deletions and replacements

20 Edit Distance f o x f o x f f a a x x insertions and deletions insertions, deletions and replacements d ij = 8 d i 1,j >< 8 1 a j = b i >< d i 1,j + w del (b i ) 1, for 1 apple i apple m, 1 apple j apple n. min d i,j 1 + w ins (a j ) 1 a j 6= b i >: >: d i 1,j 1 + w sub (a j,b i ) 1 or 2

21 Dynamic Time Warping DTW aligns two sequences of feature vectors by warping the time axis iteratively until an optimal match (according to a suitable metrics) between the two sequences is found. x 1,x 2,...x n y 1,y 2,...y n d(i, j) =c(i, j)+min 8 >< d(i 1,j), d(i 1,j 1), >: d(i, j 1)

22 Burst Detection Bursts are rare but extremely beneficial in time-series Used in number of applications Twitter: Trending topics Stock markets: Trending Stocks Text Mining: finding important time periods Elastic burst detection: Stream of data Quadratic computations not allowed

23 Burst Detection Global Average Moving Average Damped Average

24 Elastic Burst Detection Given a time-series {xi} A set of window sizes W A monotonic, associative aggregation function A which maps a sequence of values to a number. E.g. Average, Max and Thresholds associated with each window size w, f(w) Find all pairs (t,w) such that t time a time point and w is a window size in W A[x t x t+w 1 ] f(w).

25 Burst Detection - Shifted Binary Tree Level 4 Level 3 Level 2 base level shifted level Level 1 Level 0 Whenever more than f(2 + 2i 1) events are found in a window of size 2i+1, then a detailed search must be performed to check if some subwindow of size w, 2+2i 1 w 1+2i, has f(w) events.

26 Summary Periodicity of Events Auto-correlation, Periodograms and their combinations Burst Detection Techniques and elastic detection Matching of time series Euclidean matching Dynamic Time Warping

27 References On Periodicity Detection and Structural Periodic Similarity, Michail Vlachos, Philip Yu, Vittorio Castelli Burst Detection in Hierarchical Streams. Jon Kleinberg Everything you know about Dynamic Time Warping is wrong Eamonn Keogh

28 Projects Temporal and Phrase-based Indexing - Temporal Retrieval Models - Jaspreet Temporal Query Autocompletion- Avishek Crawling for Temporal Collections - Gerhard Temporal Query Suggestions - Helge

### Module II: Multimedia Data Mining

ALMA MATER STUDIORUM - UNIVERSITÀ DI BOLOGNA Module II: Multimedia Data Mining Laurea Magistrale in Ingegneria Informatica University of Bologna Multimedia Data Retrieval Home page: http://www-db.disi.unibo.it/courses/dm/

### DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

### A Secure Online Reputation Defense System from Unfair Ratings using Anomaly Detections

A Secure Online Reputation Defense System from Unfair Ratings using Anomaly Detections Asha baby PG Scholar,Department of CSE A. Kumaresan Professor, Department of CSE K. Vijayakumar Professor, Department

### Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

### EE4512 Analog and Digital Communications EE4513 Analog and Digital Communications Laboratory Dr. Dennis Silage

EE4512 Analog and Digital Communications EE4513 Analog and Digital Communications Laboratory Dr. Dennis Silage silage@temple.edu Course syllabus Course textbooks Course grades Course objectives EE4512

### Speech Recognition of Spoken Digits

Institute of Integrated Sensor Systems Dept. of Electrical Engineering and Information Technology Speech Recognition of Spoken Digits Stefanie Peters May 10, 2006 Lecture Information Sensor Signal Processing

### NRZ Bandwidth - HF Cutoff vs. SNR

Application Note: HFAN-09.0. Rev.2; 04/08 NRZ Bandwidth - HF Cutoff vs. SNR Functional Diagrams Pin Configurations appear at end of data sheet. Functional Diagrams continued at end of data sheet. UCSP

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen

CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 3: DATA TRANSFORMATION AND DIMENSIONALITY REDUCTION Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major

### Time series databases. Indexing Time Series. Time series data. Time series are ubiquitous

Time series databases Indexing Time Series A time series is a sequence of real numbers, representing the measurements of a real variable at equal time intervals Stock prices Volume of sales over time Daily

### Example: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering

Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? K-Means Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar

### Temporal Data Mining for Small and Big Data. Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant

Temporal Data Mining for Small and Big Data Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant What is Temporal Data Mining? Knowledge discovery in data that contain temporal information.

### An Order-Invariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis]

An Order-Invariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis] Stephan Spiegel and Sahin Albayrak DAI-Lab, Technische Universität Berlin, Ernst-Reuter-Platz 7,

### Signal Detection. Outline. Detection Theory. Example Applications of Detection Theory

Outline Signal Detection M. Sami Fadali Professor of lectrical ngineering University of Nevada, Reno Hypothesis testing. Neyman-Pearson (NP) detector for a known signal in white Gaussian noise (WGN). Matched

### SOFTWARE FOR GENERATION OF SPECTRUM COMPATIBLE TIME HISTORY

3 th World Conference on Earthquake Engineering Vancouver, B.C., Canada August -6, 24 Paper No. 296 SOFTWARE FOR GENERATION OF SPECTRUM COMPATIBLE TIME HISTORY ASHOK KUMAR SUMMARY One of the important

### Analysis/resynthesis with the short time Fourier transform

Analysis/resynthesis with the short time Fourier transform summer 2006 lecture on analysis, modeling and transformation of audio signals Axel Röbel Institute of communication science TU-Berlin IRCAM Analysis/Synthesis

### Chapter ML:XI (continued)

Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained

### Security and protection of digital images by using watermarking methods

Security and protection of digital images by using watermarking methods Andreja Samčović Faculty of Transport and Traffic Engineering University of Belgrade, Serbia Gjovik, june 2014. Digital watermarking

### Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

### Image Processing with. ImageJ. Biology. Imaging

Image Processing with ImageJ 1. Spatial filters Outlines background correction image denoising edges detection 2. Fourier domain filtering correction of periodic artefacts 3. Binary operations masks morphological

Fundamentals Series Analog vs. Digital Polycom, Inc. All rights reserved. Fundamentals Series Signals H.323 Analog vs. Digital SIP Defining Quality Standards Network Communication I Network Communication

### Clustering. Adrian Groza. Department of Computer Science Technical University of Cluj-Napoca

Clustering Adrian Groza Department of Computer Science Technical University of Cluj-Napoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 K-means 3 Hierarchical Clustering What is Datamining?

### MPEG, the MP3 Standard, and Audio Compression

MPEG, the MP3 Standard, and Audio Compression Mark ilgore and Jamie Wu Mathematics of the Information Age September 16, 23 Audio Compression Basic Audio Coding. Why beneficial to compress? Lossless versus

### Data Mining on Streams

Data Mining on Streams Using Decision Trees CS 536: Machine Learning Instructor: Michael Littman TA: Yihua Wu Outline Introduction to data streams Overview of traditional DT learning ALG DT learning ALGs

### Data Communications Prof. Ajit Pal Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur Lecture # 03 Data and Signal

(Refer Slide Time: 00:01:23) Data Communications Prof. Ajit Pal Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur Lecture # 03 Data and Signal Hello viewers welcome

### Decision Trees from large Databases: SLIQ

Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### Digital Image Processing

Digital Image Processing Using MATLAB Second Edition Rafael C. Gonzalez University of Tennessee Richard E. Woods MedData Interactive Steven L. Eddins The MathWorks, Inc. Gatesmark Publishing A Division

### 568 Subject Index. exponential distribution, 106, 108, 539 extension field, 545, 551 eye diagram, 193

SUBJECT INDEX additive system, 182 additive white Gaussian noise, 99, 103 aliasing, 52, 54 amplifier AM/AM characteristic, 332, 342 AM/PM characteristic, 332, 342 nonlinear, 344 two-box model, 344 amplifiers,

### Advanced Signal Processing and Digital Noise Reduction

Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New

### Univariate and Multivariate Methods PEARSON. Addison Wesley

Time Series Analysis Univariate and Multivariate Methods SECOND EDITION William W. S. Wei Department of Statistics The Fox School of Business and Management Temple University PEARSON Addison Wesley Boston

### IC05 Introduction on Networks &Visualization Nov. 2009. <mathieu.bastian@gmail.com>

IC05 Introduction on Networks &Visualization Nov. 2009 Overview 1. Networks Introduction Networks across disciplines Properties Models 2. Visualization InfoVis Data exploration

### Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

### Instant SQL Programming

Instant SQL Programming Joe Celko Wrox Press Ltd. INSTANT Table of Contents Introduction 1 What Can SQL Do for Me? 2 Who Should Use This Book? 2 How To Use This Book 3 What You Should Know 3 Conventions

### Jitter Measurements in Serial Data Signals

Jitter Measurements in Serial Data Signals Michael Schnecker, Product Manager LeCroy Corporation Introduction The increasing speed of serial data transmission systems places greater importance on measuring

### Monitoring of Internet traffic and applications

Monitoring of Internet traffic and applications Chadi BARAKAT INRIA Sophia Antipolis, France Planète research group ETH Zurich October 2009 Email: Chadi.Barakat@sophia.inria.fr WEB: http://www.inria.fr/planete/chadi

### CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

### Clustering and Data Mining in R

Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches

### 56 Mining Time Series Data

56 Mining Time Series Data Chotirat Ann Ratanamahatana 1, Jessica Lin 1, Dimitrios Gunopulos 1, Eamonn Keogh 1, Michail Vlachos 2, and Gautam Das 3 1 University of California, Riverside 2 IBM T.J. Watson

### Lecture 1-6: Noise and Filters

Lecture 1-6: Noise and Filters Overview 1. Periodic and Aperiodic Signals Review: by periodic signals, we mean signals that have a waveform shape that repeats. The time taken for the waveform to repeat

### Purpose of Time Series Analysis. Autocovariance Function. Autocorrelation Function. Part 3: Time Series I

Part 3: Time Series I Purpose of Time Series Analysis (Figure from Panofsky and Brier 1968) Autocorrelation Function Harmonic Analysis Spectrum Analysis Data Window Significance Tests Some major purposes

### NYT crossword puzzle solver

NYT crossword puzzle solver 5. Mai 2016 1 NYT crossword puzzle solver 2 NYT crossword puzzle solver 1 Problem Description 2 Concept of Solution 3 Grid extraction 4 Box Classification 5 Solve puzzle 6 Results

### TCOM 370 NOTES 99-4 BANDWIDTH, FREQUENCY RESPONSE, AND CAPACITY OF COMMUNICATION LINKS

TCOM 370 NOTES 99-4 BANDWIDTH, FREQUENCY RESPONSE, AND CAPACITY OF COMMUNICATION LINKS 1. Bandwidth: The bandwidth of a communication link, or in general any system, was loosely defined as the width of

### RADIO FREQUENCY INTERFERENCE AND CAPACITY REDUCTION IN DSL

RADIO FREQUENCY INTERFERENCE AND CAPACITY REDUCTION IN DSL Padmabala Venugopal, Michael J. Carter*, Scott A. Valcourt, InterOperability Laboratory, Technology Drive Suite, University of New Hampshire,

### THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS

THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS O.U. Sezerman 1, R. Islamaj 2, E. Alpaydin 2 1 Laborotory of Computational Biology, Sabancı University, Istanbul, Turkey. 2 Computer Engineering

### MINING TIME SERIES DATA

Chapter 1 MINING TIME SERIES DATA Chotirat Ann Ratanamahatana, Jessica Lin, Dimitrios Gunopulos, Eamonn Keogh University of California, Riverside Michail Vlachos IBM T.J. Watson Research Center Gautam

### Broadband Networks. Prof. Dr. Abhay Karandikar. Electrical Engineering Department. Indian Institute of Technology, Bombay. Lecture - 29.

Broadband Networks Prof. Dr. Abhay Karandikar Electrical Engineering Department Indian Institute of Technology, Bombay Lecture - 29 Voice over IP So, today we will discuss about voice over IP and internet

### Copyright 2007 Casa Software Ltd. www.casaxps.com. ToF Mass Calibration

ToF Mass Calibration Essentially, the relationship between the mass m of an ion and the time taken for the ion of a given charge to travel a fixed distance is quadratic in the flight time t. For an ideal

### Error Log Processing for Accurate Failure Prediction. Humboldt-Universität zu Berlin

Error Log Processing for Accurate Failure Prediction Felix Salfner ICSI Berkeley Steffen Tschirpke Humboldt-Universität zu Berlin Introduction Context of work: Error-based online failure prediction: error

### Computer Networks and Internets, 5e Chapter 6 Information Sources and Signals. Introduction

Computer Networks and Internets, 5e Chapter 6 Information Sources and Signals Modified from the lecture slides of Lami Kaya (LKaya@ieee.org) for use CECS 474, Fall 2008. 2009 Pearson Education Inc., Upper

### ANALYZER BASICS WHAT IS AN FFT SPECTRUM ANALYZER? 2-1

WHAT IS AN FFT SPECTRUM ANALYZER? ANALYZER BASICS The SR760 FFT Spectrum Analyzer takes a time varying input signal, like you would see on an oscilloscope trace, and computes its frequency spectrum. Fourier's

### Image Compression through DCT and Huffman Coding Technique

International Journal of Current Engineering and Technology E-ISSN 2277 4106, P-ISSN 2347 5161 2015 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Research Article Rahul

### Email: tjohn@mail.nplindia.ernet.in

USE OF VIRTUAL INSTRUMENTS IN RADIO AND ATMOSPHERIC EXPERIMENTS P.N. VIJAYAKUMAR, THOMAS JOHN AND S.C. GARG RADIO AND ATMOSPHERIC SCIENCE DIVISION, NATIONAL PHYSICAL LABORATORY, NEW DELHI 110012, INDIA

### Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication

Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Thomas Reilly Data Physics Corporation 1741 Technology Drive, Suite 260 San Jose, CA 95110 (408) 216-8440 This paper

### Machine Learning using MapReduce

Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

### Fast Fourier Transforms and Power Spectra in LabVIEW

Application Note 4 Introduction Fast Fourier Transforms and Power Spectra in LabVIEW K. Fahy, E. Pérez Ph.D. The Fourier transform is one of the most powerful signal analysis tools, applicable to a wide

### The counterpart to a DAC is the ADC, which is generally a more complicated circuit. One of the most popular ADC circuit is the successive

The counterpart to a DAC is the ADC, which is generally a more complicated circuit. One of the most popular ADC circuit is the successive approximation converter. 1 2 The idea of sampling is fully covered

### The Scientific Data Mining Process

Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### The Calculation of G rms

The Calculation of G rms QualMark Corp. Neill Doertenbach The metric of G rms is typically used to specify and compare the energy in repetitive shock vibration systems. However, the method of arriving

### HES-SO Master of Science in Engineering. Clustering. Prof. Laura Elena Raileanu HES-SO Yverdon-les-Bains (HEIG-VD)

HES-SO Master of Science in Engineering Clustering Prof. Laura Elena Raileanu HES-SO Yverdon-les-Bains (HEIG-VD) Plan Motivation Hierarchical Clustering K-Means Clustering 2 Problem Setup Arrange items

### School Class Monitoring System Based on Audio Signal Processing

C. R. Rashmi 1,,C.P.Shantala 2 andt.r.yashavanth 3 1 Department of CSE, PG Student, CIT, Gubbi, Tumkur, Karnataka, India. 2 Department of CSE, Vice Principal & HOD, CIT, Gubbi, Tumkur, Karnataka, India.

### Myanmar Continuous Speech Recognition System Based on DTW and HMM

Myanmar Continuous Speech Recognition System Based on DTW and HMM Ingyin Khaing Department of Information and Technology University of Technology (Yatanarpon Cyber City),near Pyin Oo Lwin, Myanmar Abstract-

### Practical Graph Mining with R. 5. Link Analysis

Practical Graph Mining with R 5. Link Analysis Outline Link Analysis Concepts Metrics for Analyzing Networks PageRank HITS Link Prediction 2 Link Analysis Concepts Link A relationship between two entities

### Speech Signal Processing: An Overview

Speech Signal Processing: An Overview S. R. M. Prasanna Department of Electronics and Electrical Engineering Indian Institute of Technology Guwahati December, 2012 Prasanna (EMST Lab, EEE, IITG) Speech

### Chapter 20: Data Analysis

Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification

### T = 1 f. Phase. Measure of relative position in time within a single period of a signal For a periodic signal f(t), phase is fractional part t p

Data Transmission Concepts and terminology Transmission terminology Transmission from transmitter to receiver goes over some transmission medium using electromagnetic waves Guided media. Waves are guided

### Alignment and Preprocessing for Data Analysis

Alignment and Preprocessing for Data Analysis Preprocessing tools for chromatography Basics of alignment GC FID (D) data and issues PCA F Ratios GC MS (D) data and issues PCA F Ratios PARAFAC Piecewise

### Data Mining 5. Cluster Analysis

Data Mining 5. Cluster Analysis 5.2 Fall 2009 Instructor: Dr. Masoud Yaghini Outline Data Structures Interval-Valued (Numeric) Variables Binary Variables Categorical Variables Ordinal Variables Variables

### Anomaly Detection and Predictive Maintenance

Anomaly Detection and Predictive Maintenance Rosaria Silipo Iris Adae Christian Dietz Phil Winters Rosaria.Silipo@knime.com Iris.Adae@uni-konstanz.de Christian.Dietz@uni-konstanz.de Phil.Winters@knime.com

### Chapter 10 Introduction to Time Series Analysis

Chapter 1 Introduction to Time Series Analysis A time series is a collection of observations made sequentially in time. Examples are daily mortality counts, particulate air pollution measurements, and

### Local outlier detection in data forensics: data mining approach to flag unusual schools

Local outlier detection in data forensics: data mining approach to flag unusual schools Mayuko Simon Data Recognition Corporation Paper presented at the 2012 Conference on Statistical Detection of Potential

### Evaluating System Suitability CE, GC, LC and A/D ChemStation Revisions: A.03.0x- A.08.0x

CE, GC, LC and A/D ChemStation Revisions: A.03.0x- A.08.0x This document is believed to be accurate and up-to-date. However, Agilent Technologies, Inc. cannot assume responsibility for the use of this

### Unsupervised learning: Clustering

Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What

### Outline. Random Variables. Examples. Random Variable

Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,

### Notes: Chapter 2 Section 2.2: Proof by Induction

Notes: Chapter 2 Section 2.2: Proof by Induction Basic Induction. To prove: n, a W, n a, S n. (1) Prove the base case - S a. (2) Let k a and prove that S k S k+1 Example 1. n N, n i = n(n+1) 2. Example

### Solutions to Exam in Speech Signal Processing EN2300

Solutions to Exam in Speech Signal Processing EN23 Date: Thursday, Dec 2, 8: 3: Place: Allowed: Grades: Language: Solutions: Q34, Q36 Beta Math Handbook (or corresponding), calculator with empty memory.

### Lezione 6 Communications Blockset

Corso di Tecniche CAD per le Telecomunicazioni A.A. 2007-2008 Lezione 6 Communications Blockset Ing. Marco GALEAZZI 1 What Is Communications Blockset? Communications Blockset extends Simulink with a comprehensive

### 0.1 What is Cluster Analysis?

Cluster Analysis 1 2 0.1 What is Cluster Analysis? Cluster analysis is concerned with forming groups of similar objects based on several measurements of different kinds made on the objects. The key idea

### Exercises with solutions (1)

Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal

### Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

### ANALYTICS IN BIG DATA ERA

ANALYTICS IN BIG DATA ERA ANALYTICS TECHNOLOGY AND ARCHITECTURE TO MANAGE VELOCITY AND VARIETY, DISCOVER RELATIONSHIPS AND CLASSIFY HUGE AMOUNT OF DATA MAURIZIO SALUSTI SAS Copyr i g ht 2012, SAS Ins titut

### Available from Deakin Research Online:

This is the authors final peered reviewed (post print) version of the item published as: Adibi,S 2014, A low overhead scaled equalized harmonic-based voice authentication system, Telematics and informatics,

### Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

### Command-induced Tracking Jitter Study I D. Clark November 24, 2009

Command-induced Tracking Jitter Study I D. Clark November 24, 2009 Introduction Reports of excessive tracking jitter on the MMT elevation axis have lately been theorized to be caused by the input command

### L15: statistical clustering

Similarity measures Criterion functions Cluster validity Flat clustering algorithms k-means ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern

### Alternative Biometric as Method of Information Security of Healthcare Systems

Alternative Biometric as Method of Information Security of Healthcare Systems Ekaterina Andreeva Saint-Petersburg State University of Aerospace Instrumentation Saint-Petersburg, Russia eandreeva89@gmail.com

### A Learning Based Method for Super-Resolution of Low Resolution Images

A Learning Based Method for Super-Resolution of Low Resolution Images Emre Ugur June 1, 2004 emre.ugur@ceng.metu.edu.tr Abstract The main objective of this project is the study of a learning based method

### Spot me if you can: Uncovering spoken phrases in encrypted VoIP conversations

Spot me if you can: Uncovering spoken phrases in encrypted VoIP conversations C. Wright, L. Ballard, S. Coull, F. Monrose, G. Masson Talk held by Goran Doychev Selected Topics in Information Security and

### Digital Modulation. David Tipper. Department of Information Science and Telecommunications University of Pittsburgh. Typical Communication System

Digital Modulation David Tipper Associate Professor Department of Information Science and Telecommunications University of Pittsburgh http://www.tele.pitt.edu/tipper.html Typical Communication System Source

### 4.3 Analog-to-Digital Conversion

4.3 Analog-to-Digital Conversion overview including timing considerations block diagram of a device using a DAC and comparator example of a digitized spectrum number of data points required to describe

### IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH

IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH Kalinka Mihaylova Kaloyanova St. Kliment Ohridski University of Sofia, Faculty of Mathematics and Informatics Sofia 1164, Bulgaria

### Investigating Clinical Care Pathways Correlated with Outcomes

Investigating Clinical Care Pathways Correlated with Outcomes Geetika T. Lakshmanan, Szabolcs Rozsnyai, Fei Wang IBM T. J. Watson Research Center, NY, USA August 2013 Outline Care Pathways Typical Challenges

### Frequency Domain Characterization of Signals. Yao Wang Polytechnic University, Brooklyn, NY11201 http: //eeweb.poly.edu/~yao

Frequency Domain Characterization of Signals Yao Wang Polytechnic University, Brooklyn, NY1121 http: //eeweb.poly.edu/~yao Signal Representation What is a signal Time-domain description Waveform representation

### Private Record Linkage with Bloom Filters

To appear in: Proceedings of Statistics Canada Symposium 2010 Social Statistics: The Interplay among Censuses, Surveys and Administrative Data Private Record Linkage with Bloom Filters Rainer Schnell,

### DIGITAL-TO-ANALOGUE AND ANALOGUE-TO-DIGITAL CONVERSION

DIGITAL-TO-ANALOGUE AND ANALOGUE-TO-DIGITAL CONVERSION Introduction The outputs from sensors and communications receivers are analogue signals that have continuously varying amplitudes. In many systems

### Data Mining and Visualization

Data Mining and Visualization Jeremy Walton NAG Ltd, Oxford Overview Data mining components Functionality Example application Quality control Visualization Use of 3D Example application Market research

### E-navigation, from sensors to ship behaviour analysis

E-navigation, from sensors to ship behaviour analysis Laurent ETIENNE, Loïc SALMON French Naval Academy Research Institute Geographic Information Systems Group laurent.etienne@ecole-navale.fr loic.salmon@ecole-navale.fr