Foundations  2. Periodicity Detection, Timeseries Correlation, Burst Detection. Temporal Information Retrieval


 Arnold Conley
 11 months ago
 Views:
Transcription
1 Foundations  2 Periodicity Detection, Timeseries Correlation, Burst Detection Temporal Information Retrieval
2 Time Series An ordered sequence of values (data points) of variables at equally spaced time intervals
3 Periodicity Detection How does one identify periodic values
4 Periodicity Detection Timeseries is in the time domain Method1 (DFT): Identify the underlying periodic patterns by transforming into the frequency domain Method 2 (Autocorrelation) Correlate the signal with itself Find dominant frequencies
5 Fourier Transform A signal has an amplitude (strength), frequency (periodicity) and phase (offset) phase frequency amplitude Fourier Transform converts a signal from the time domain to the frequency domain
6 Discrete Fourier Transform (DFT) A Fourier analysis is a method for expressing a function as a sum of periodic components, and for recovering the function from those components. When both the function and its Fourier transform are replaced with discretized counterparts, it is called the discrete Fourier transform (DFT). fourier transform f 4 f 3 f 2 f 1 f 0 inv. fourier transform Advantages of DFT apart from periodicity detection? denoising, compression
7 Discrete Fourier Transform (DFT) fourier coefficients fourier transform F[n] periodogram f (Hz) sinusoid X(f k/n )= p 1 NX 1 N n=0 x(n) e j2 kn N The fourier coefficients encode both the amplitude and phase
8 Power Spectral Density (PSD) Estimation fourier transform F[n] f (Hz) To find out the dominant frequency we need to find the power at each frequency Periodogram encodes the strength at a given frequency P(f k/n )= X(f k/n ) 2 k =0, 1... N 1 2
9 PSD estimation using Periodogram time series data or signal P2= P1= 7 periodogram P(f k/n )= X(f k/n ) 2 k =0, 1... N To find the dominant frequencies choose the topk dominant frequencies
10 Disadvantages of the Periodogram time series data or signal P2= P1= 7 periodogram P(f k/n )= X(f k/n ) 2 k =0, 1... N Good only for short and medium periodicities Spectral leakage  frequencies not integral multiples of the DFT bin spread over other bins false alarms
11 Autocorrelation Correlate the time series with itself Peaks get amplified ACF ( ) = P N i=1 Y i Y i+ N Finegrained periodicity detector
12 Autocorrelation time series data or signal 0 x day Autocorrelation ACF ( ) = P N i=1 Y i Y i+ N To determine dominant period significance threshold needs to be specified Multiples of the same period are also peaks needs post processing
13 AutoPeriod Autocorrelation : Good for large periods but difficult to automatically determine periods Periodogram : Easy to threshold but not accurate for short periods Idea: Get candidate periods from Periodogram and validate false alarms using Autocorrelation hill Sequence Periodogram Candidate Autocorrelation Refine Period Period Periods valley False Alarm Dismiss
14 AutoPeriod P1= P2= P3= Periodogram Circular Autocorrelation Valley: False Alarms Hill: Candidate Period 357: correct period
15 Matching Time Series Similar time series suggest similar things
16 Matching Time Series Correlating time series used for clustering, classification, anomaly detection, speech recognition etc.
17 Matching Time Series What measure would you use to match two time series? d = X t y t x t Euclidean Distance Why is Euclidean matching not good enough?
18 Dynamic Time Warping Time series might be shifted Time series might be compressed at some point in time Random noise at some points Dynamic time warping measures the distance between two sequences under certain restrictions. Not a metric. Triangle inequality doesn't hold
19 Detour  Edit Distance Edit distance measures how many steps it takes to convert a string to another based on restrictions Restrictions define cost function insertion, deletion, replacement f o x f o x f f a a x x insertions and deletions insertions, deletions and replacements
20 Edit Distance f o x f o x f f a a x x insertions and deletions insertions, deletions and replacements d ij = 8 d i 1,j >< 8 1 a j = b i >< d i 1,j + w del (b i ) 1, for 1 apple i apple m, 1 apple j apple n. min d i,j 1 + w ins (a j ) 1 a j 6= b i >: >: d i 1,j 1 + w sub (a j,b i ) 1 or 2
21 Dynamic Time Warping DTW aligns two sequences of feature vectors by warping the time axis iteratively until an optimal match (according to a suitable metrics) between the two sequences is found. x 1,x 2,...x n y 1,y 2,...y n d(i, j) =c(i, j)+min 8 >< d(i 1,j), d(i 1,j 1), >: d(i, j 1)
22 Burst Detection Bursts are rare but extremely beneficial in timeseries Used in number of applications Twitter: Trending topics Stock markets: Trending Stocks Text Mining: finding important time periods Elastic burst detection: Stream of data Quadratic computations not allowed
23 Burst Detection Global Average Moving Average Damped Average
24 Elastic Burst Detection Given a timeseries {xi} A set of window sizes W A monotonic, associative aggregation function A which maps a sequence of values to a number. E.g. Average, Max and Thresholds associated with each window size w, f(w) Find all pairs (t,w) such that t time a time point and w is a window size in W A[x t x t+w 1 ] f(w).
25 Burst Detection  Shifted Binary Tree Level 4 Level 3 Level 2 base level shifted level Level 1 Level 0 Whenever more than f(2 + 2i 1) events are found in a window of size 2i+1, then a detailed search must be performed to check if some subwindow of size w, 2+2i 1 w 1+2i, has f(w) events.
26 Summary Periodicity of Events Autocorrelation, Periodograms and their combinations Burst Detection Techniques and elastic detection Matching of time series Euclidean matching Dynamic Time Warping
27 References On Periodicity Detection and Structural Periodic Similarity, Michail Vlachos, Philip Yu, Vittorio Castelli Burst Detection in Hierarchical Streams. Jon Kleinberg Everything you know about Dynamic Time Warping is wrong Eamonn Keogh
28 Projects Temporal and Phrasebased Indexing  Temporal Retrieval Models  Jaspreet Temporal Query Autocompletion Avishek Crawling for Temporal Collections  Gerhard Temporal Query Suggestions  Helge
Module II: Multimedia Data Mining
ALMA MATER STUDIORUM  UNIVERSITÀ DI BOLOGNA Module II: Multimedia Data Mining Laurea Magistrale in Ingegneria Informatica University of Bologna Multimedia Data Retrieval Home page: http://wwwdb.disi.unibo.it/courses/dm/
More informationDATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS
DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDDLAB ISTI CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar
More informationA Secure Online Reputation Defense System from Unfair Ratings using Anomaly Detections
A Secure Online Reputation Defense System from Unfair Ratings using Anomaly Detections Asha baby PG Scholar,Department of CSE A. Kumaresan Professor, Department of CSE K. Vijayakumar Professor, Department
More informationLecture 20: Clustering
Lecture 20: Clustering Wrapup of neural nets (from last lecture Introduction to unsupervised learning Kmeans clustering COMP424, Lecture 20  April 3, 2013 1 Unsupervised learning In supervised learning,
More informationEE4512 Analog and Digital Communications EE4513 Analog and Digital Communications Laboratory Dr. Dennis Silage
EE4512 Analog and Digital Communications EE4513 Analog and Digital Communications Laboratory Dr. Dennis Silage silage@temple.edu Course syllabus Course textbooks Course grades Course objectives EE4512
More informationSpeech Recognition of Spoken Digits
Institute of Integrated Sensor Systems Dept. of Electrical Engineering and Information Technology Speech Recognition of Spoken Digits Stefanie Peters May 10, 2006 Lecture Information Sensor Signal Processing
More informationNRZ Bandwidth  HF Cutoff vs. SNR
Application Note: HFAN09.0. Rev.2; 04/08 NRZ Bandwidth  HF Cutoff vs. SNR Functional Diagrams Pin Configurations appear at end of data sheet. Functional Diagrams continued at end of data sheet. UCSP
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationCS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 3: DATA TRANSFORMATION AND DIMENSIONALITY REDUCTION Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major
More informationTime series databases. Indexing Time Series. Time series data. Time series are ubiquitous
Time series databases Indexing Time Series A time series is a sequence of real numbers, representing the measurements of a real variable at equal time intervals Stock prices Volume of sales over time Daily
More informationExample: Document Clustering. Clustering: Definition. Notion of a Cluster can be Ambiguous. Types of Clusterings. Hierarchical Clustering
Overview Prognostic Models and Data Mining in Medicine, part I Cluster Analsis What is Cluster Analsis? KMeans Clustering Hierarchical Clustering Cluster Validit Eample: Microarra data analsis 6 Summar
More informationTemporal Data Mining for Small and Big Data. Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant
Temporal Data Mining for Small and Big Data Theophano Mitsa, Ph.D. Independent Data Mining/Analytics Consultant What is Temporal Data Mining? Knowledge discovery in data that contain temporal information.
More informationAn OrderInvariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis]
An OrderInvariant Time Series Distance Measure [Position on Recent Developments in Time Series Analysis] Stephan Spiegel and Sahin Albayrak DAILab, Technische Universität Berlin, ErnstReuterPlatz 7,
More informationSignal Detection. Outline. Detection Theory. Example Applications of Detection Theory
Outline Signal Detection M. Sami Fadali Professor of lectrical ngineering University of Nevada, Reno Hypothesis testing. NeymanPearson (NP) detector for a known signal in white Gaussian noise (WGN). Matched
More informationSOFTWARE FOR GENERATION OF SPECTRUM COMPATIBLE TIME HISTORY
3 th World Conference on Earthquake Engineering Vancouver, B.C., Canada August 6, 24 Paper No. 296 SOFTWARE FOR GENERATION OF SPECTRUM COMPATIBLE TIME HISTORY ASHOK KUMAR SUMMARY One of the important
More informationAnalysis/resynthesis with the short time Fourier transform
Analysis/resynthesis with the short time Fourier transform summer 2006 lecture on analysis, modeling and transformation of audio signals Axel Röbel Institute of communication science TUBerlin IRCAM Analysis/Synthesis
More informationChapter ML:XI (continued)
Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis DensityBased Cluster Analysis Cluster Evaluation Constrained
More informationSecurity and protection of digital images by using watermarking methods
Security and protection of digital images by using watermarking methods Andreja Samčović Faculty of Transport and Traffic Engineering University of Belgrade, Serbia Gjovik, june 2014. Digital watermarking
More informationPolynomials and the Fast Fourier Transform (FFT) Battle Plan
Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and pointvalue representation
More informationImage Processing with. ImageJ. Biology. Imaging
Image Processing with ImageJ 1. Spatial filters Outlines background correction image denoising edges detection 2. Fourier domain filtering correction of periodic artefacts 3. Binary operations masks morphological
More informationFundamentals Series Analog vs. Digital. Polycom, Inc. All rights reserved.
Fundamentals Series Analog vs. Digital Polycom, Inc. All rights reserved. Fundamentals Series Signals H.323 Analog vs. Digital SIP Defining Quality Standards Network Communication I Network Communication
More informationClustering. Adrian Groza. Department of Computer Science Technical University of ClujNapoca
Clustering Adrian Groza Department of Computer Science Technical University of ClujNapoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 Kmeans 3 Hierarchical Clustering What is Datamining?
More informationMPEG, the MP3 Standard, and Audio Compression
MPEG, the MP3 Standard, and Audio Compression Mark ilgore and Jamie Wu Mathematics of the Information Age September 16, 23 Audio Compression Basic Audio Coding. Why beneficial to compress? Lossless versus
More informationData Mining on Streams
Data Mining on Streams Using Decision Trees CS 536: Machine Learning Instructor: Michael Littman TA: Yihua Wu Outline Introduction to data streams Overview of traditional DT learning ALG DT learning ALGs
More informationData Communications Prof. Ajit Pal Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur Lecture # 03 Data and Signal
(Refer Slide Time: 00:01:23) Data Communications Prof. Ajit Pal Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur Lecture # 03 Data and Signal Hello viewers welcome
More informationDecision Trees from large Databases: SLIQ
Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values
More informationMedical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu
Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?
More informationDigital Image Processing
Digital Image Processing Using MATLAB Second Edition Rafael C. Gonzalez University of Tennessee Richard E. Woods MedData Interactive Steven L. Eddins The MathWorks, Inc. Gatesmark Publishing A Division
More information568 Subject Index. exponential distribution, 106, 108, 539 extension field, 545, 551 eye diagram, 193
SUBJECT INDEX additive system, 182 additive white Gaussian noise, 99, 103 aliasing, 52, 54 amplifier AM/AM characteristic, 332, 342 AM/PM characteristic, 332, 342 nonlinear, 344 twobox model, 344 amplifiers,
More informationAdvanced Signal Processing and Digital Noise Reduction
Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New
More informationUnivariate and Multivariate Methods PEARSON. Addison Wesley
Time Series Analysis Univariate and Multivariate Methods SECOND EDITION William W. S. Wei Department of Statistics The Fox School of Business and Management Temple University PEARSON Addison Wesley Boston
More information
IC05 Introduction on Networks &Visualization Nov. 2009.
IC05 Introduction on Networks &Visualization Nov. 2009 Overview 1. Networks Introduction Networks across disciplines Properties Models 2. Visualization InfoVis Data exploration
More informationClustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016
Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with
More informationInstant SQL Programming
Instant SQL Programming Joe Celko Wrox Press Ltd. INSTANT Table of Contents Introduction 1 What Can SQL Do for Me? 2 Who Should Use This Book? 2 How To Use This Book 3 What You Should Know 3 Conventions
More informationJitter Measurements in Serial Data Signals
Jitter Measurements in Serial Data Signals Michael Schnecker, Product Manager LeCroy Corporation Introduction The increasing speed of serial data transmission systems places greater importance on measuring
More informationMonitoring of Internet traffic and applications
Monitoring of Internet traffic and applications Chadi BARAKAT INRIA Sophia Antipolis, France Planète research group ETH Zurich October 2009 Email: Chadi.Barakat@sophia.inria.fr WEB: http://www.inria.fr/planete/chadi
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More informationClustering and Data Mining in R
Clustering and Data Mining in R Workshop Supplement Thomas Girke December 10, 2011 Introduction Data Preprocessing Data Transformations Distance Methods Cluster Linkage Hierarchical Clustering Approaches
More information56 Mining Time Series Data
56 Mining Time Series Data Chotirat Ann Ratanamahatana 1, Jessica Lin 1, Dimitrios Gunopulos 1, Eamonn Keogh 1, Michail Vlachos 2, and Gautam Das 3 1 University of California, Riverside 2 IBM T.J. Watson
More informationLecture 16: Noise and Filters
Lecture 16: Noise and Filters Overview 1. Periodic and Aperiodic Signals Review: by periodic signals, we mean signals that have a waveform shape that repeats. The time taken for the waveform to repeat
More informationPurpose of Time Series Analysis. Autocovariance Function. Autocorrelation Function. Part 3: Time Series I
Part 3: Time Series I Purpose of Time Series Analysis (Figure from Panofsky and Brier 1968) Autocorrelation Function Harmonic Analysis Spectrum Analysis Data Window Significance Tests Some major purposes
More informationNYT crossword puzzle solver
NYT crossword puzzle solver 5. Mai 2016 1 NYT crossword puzzle solver 2 NYT crossword puzzle solver 1 Problem Description 2 Concept of Solution 3 Grid extraction 4 Box Classification 5 Solve puzzle 6 Results
More informationTCOM 370 NOTES 994 BANDWIDTH, FREQUENCY RESPONSE, AND CAPACITY OF COMMUNICATION LINKS
TCOM 370 NOTES 994 BANDWIDTH, FREQUENCY RESPONSE, AND CAPACITY OF COMMUNICATION LINKS 1. Bandwidth: The bandwidth of a communication link, or in general any system, was loosely defined as the width of
More informationRADIO FREQUENCY INTERFERENCE AND CAPACITY REDUCTION IN DSL
RADIO FREQUENCY INTERFERENCE AND CAPACITY REDUCTION IN DSL Padmabala Venugopal, Michael J. Carter*, Scott A. Valcourt, InterOperability Laboratory, Technology Drive Suite, University of New Hampshire,
More informationTHREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC TERISTICS
THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC TERISTICS O.U. Sezerman 1, R. Islamaj 2, E. Alpaydin 2 1 Laborotory of Computational Biology, Sabancı University, Istanbul, Turkey. 2 Computer Engineering
More informationMINING TIME SERIES DATA
Chapter 1 MINING TIME SERIES DATA Chotirat Ann Ratanamahatana, Jessica Lin, Dimitrios Gunopulos, Eamonn Keogh University of California, Riverside Michail Vlachos IBM T.J. Watson Research Center Gautam
More informationBroadband Networks. Prof. Dr. Abhay Karandikar. Electrical Engineering Department. Indian Institute of Technology, Bombay. Lecture  29.
Broadband Networks Prof. Dr. Abhay Karandikar Electrical Engineering Department Indian Institute of Technology, Bombay Lecture  29 Voice over IP So, today we will discuss about voice over IP and internet
More informationCopyright 2007 Casa Software Ltd. www.casaxps.com. ToF Mass Calibration
ToF Mass Calibration Essentially, the relationship between the mass m of an ion and the time taken for the ion of a given charge to travel a fixed distance is quadratic in the flight time t. For an ideal
More informationError Log Processing for Accurate Failure Prediction. HumboldtUniversität zu Berlin
Error Log Processing for Accurate Failure Prediction Felix Salfner ICSI Berkeley Steffen Tschirpke HumboldtUniversität zu Berlin Introduction Context of work: Errorbased online failure prediction: error
More informationComputer Networks and Internets, 5e Chapter 6 Information Sources and Signals. Introduction
Computer Networks and Internets, 5e Chapter 6 Information Sources and Signals Modified from the lecture slides of Lami Kaya (LKaya@ieee.org) for use CECS 474, Fall 2008. 2009 Pearson Education Inc., Upper
More informationANALYZER BASICS WHAT IS AN FFT SPECTRUM ANALYZER? 21
WHAT IS AN FFT SPECTRUM ANALYZER? ANALYZER BASICS The SR760 FFT Spectrum Analyzer takes a time varying input signal, like you would see on an oscilloscope trace, and computes its frequency spectrum. Fourier's
More informationImage Compression through DCT and Huffman Coding Technique
International Journal of Current Engineering and Technology EISSN 2277 4106, PISSN 2347 5161 2015 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Research Article Rahul
More informationEmail: tjohn@mail.nplindia.ernet.in
USE OF VIRTUAL INSTRUMENTS IN RADIO AND ATMOSPHERIC EXPERIMENTS P.N. VIJAYAKUMAR, THOMAS JOHN AND S.C. GARG RADIO AND ATMOSPHERIC SCIENCE DIVISION, NATIONAL PHYSICAL LABORATORY, NEW DELHI 110012, INDIA
More informationTime Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication
Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Thomas Reilly Data Physics Corporation 1741 Technology Drive, Suite 260 San Jose, CA 95110 (408) 2168440 This paper
More informationMachine Learning using MapReduce
Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous
More informationFast Fourier Transforms and Power Spectra in LabVIEW
Application Note 4 Introduction Fast Fourier Transforms and Power Spectra in LabVIEW K. Fahy, E. Pérez Ph.D. The Fourier transform is one of the most powerful signal analysis tools, applicable to a wide
More informationThe counterpart to a DAC is the ADC, which is generally a more complicated circuit. One of the most popular ADC circuit is the successive
The counterpart to a DAC is the ADC, which is generally a more complicated circuit. One of the most popular ADC circuit is the successive approximation converter. 1 2 The idea of sampling is fully covered
More informationThe Scientific Data Mining Process
Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More informationThe Calculation of G rms
The Calculation of G rms QualMark Corp. Neill Doertenbach The metric of G rms is typically used to specify and compare the energy in repetitive shock vibration systems. However, the method of arriving
More informationHESSO Master of Science in Engineering. Clustering. Prof. Laura Elena Raileanu HESSO YverdonlesBains (HEIGVD)
HESSO Master of Science in Engineering Clustering Prof. Laura Elena Raileanu HESSO YverdonlesBains (HEIGVD) Plan Motivation Hierarchical Clustering KMeans Clustering 2 Problem Setup Arrange items
More informationSchool Class Monitoring System Based on Audio Signal Processing
C. R. Rashmi 1,,C.P.Shantala 2 andt.r.yashavanth 3 1 Department of CSE, PG Student, CIT, Gubbi, Tumkur, Karnataka, India. 2 Department of CSE, Vice Principal & HOD, CIT, Gubbi, Tumkur, Karnataka, India.
More informationMyanmar Continuous Speech Recognition System Based on DTW and HMM
Myanmar Continuous Speech Recognition System Based on DTW and HMM Ingyin Khaing Department of Information and Technology University of Technology (Yatanarpon Cyber City),near Pyin Oo Lwin, Myanmar Abstract
More informationPractical Graph Mining with R. 5. Link Analysis
Practical Graph Mining with R 5. Link Analysis Outline Link Analysis Concepts Metrics for Analyzing Networks PageRank HITS Link Prediction 2 Link Analysis Concepts Link A relationship between two entities
More informationSpeech Signal Processing: An Overview
Speech Signal Processing: An Overview S. R. M. Prasanna Department of Electronics and Electrical Engineering Indian Institute of Technology Guwahati December, 2012 Prasanna (EMST Lab, EEE, IITG) Speech
More informationChapter 20: Data Analysis
Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.dbbook.com for conditions on reuse Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification
More informationT = 1 f. Phase. Measure of relative position in time within a single period of a signal For a periodic signal f(t), phase is fractional part t p
Data Transmission Concepts and terminology Transmission terminology Transmission from transmitter to receiver goes over some transmission medium using electromagnetic waves Guided media. Waves are guided
More informationAlignment and Preprocessing for Data Analysis
Alignment and Preprocessing for Data Analysis Preprocessing tools for chromatography Basics of alignment GC FID (D) data and issues PCA F Ratios GC MS (D) data and issues PCA F Ratios PARAFAC Piecewise
More informationData Mining 5. Cluster Analysis
Data Mining 5. Cluster Analysis 5.2 Fall 2009 Instructor: Dr. Masoud Yaghini Outline Data Structures IntervalValued (Numeric) Variables Binary Variables Categorical Variables Ordinal Variables Variables
More informationAnomaly Detection and Predictive Maintenance
Anomaly Detection and Predictive Maintenance Rosaria Silipo Iris Adae Christian Dietz Phil Winters Rosaria.Silipo@knime.com Iris.Adae@unikonstanz.de Christian.Dietz@unikonstanz.de Phil.Winters@knime.com
More informationChapter 10 Introduction to Time Series Analysis
Chapter 1 Introduction to Time Series Analysis A time series is a collection of observations made sequentially in time. Examples are daily mortality counts, particulate air pollution measurements, and
More informationLocal outlier detection in data forensics: data mining approach to flag unusual schools
Local outlier detection in data forensics: data mining approach to flag unusual schools Mayuko Simon Data Recognition Corporation Paper presented at the 2012 Conference on Statistical Detection of Potential
More informationEvaluating System Suitability CE, GC, LC and A/D ChemStation Revisions: A.03.0x A.08.0x
CE, GC, LC and A/D ChemStation Revisions: A.03.0x A.08.0x This document is believed to be accurate and uptodate. However, Agilent Technologies, Inc. cannot assume responsibility for the use of this
More informationUnsupervised learning: Clustering
Unsupervised learning: Clustering Salissou Moutari Centre for Statistical Science and Operational Research CenSSOR 17 th September 2013 Unsupervised learning: Clustering 1/52 Outline 1 Introduction What
More informationOutline. Random Variables. Examples. Random Variable
Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,
More informationNotes: Chapter 2 Section 2.2: Proof by Induction
Notes: Chapter 2 Section 2.2: Proof by Induction Basic Induction. To prove: n, a W, n a, S n. (1) Prove the base case  S a. (2) Let k a and prove that S k S k+1 Example 1. n N, n i = n(n+1) 2. Example
More informationSolutions to Exam in Speech Signal Processing EN2300
Solutions to Exam in Speech Signal Processing EN23 Date: Thursday, Dec 2, 8: 3: Place: Allowed: Grades: Language: Solutions: Q34, Q36 Beta Math Handbook (or corresponding), calculator with empty memory.
More informationLezione 6 Communications Blockset
Corso di Tecniche CAD per le Telecomunicazioni A.A. 20072008 Lezione 6 Communications Blockset Ing. Marco GALEAZZI 1 What Is Communications Blockset? Communications Blockset extends Simulink with a comprehensive
More information0.1 What is Cluster Analysis?
Cluster Analysis 1 2 0.1 What is Cluster Analysis? Cluster analysis is concerned with forming groups of similar objects based on several measurements of different kinds made on the objects. The key idea
More informationExercises with solutions (1)
Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal
More informationData Clustering. Dec 2nd, 2013 Kyrylo Bessonov
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms kmeans Hierarchical Main
More informationANALYTICS IN BIG DATA ERA
ANALYTICS IN BIG DATA ERA ANALYTICS TECHNOLOGY AND ARCHITECTURE TO MANAGE VELOCITY AND VARIETY, DISCOVER RELATIONSHIPS AND CLASSIFY HUGE AMOUNT OF DATA MAURIZIO SALUSTI SAS Copyr i g ht 2012, SAS Ins titut
More informationAvailable from Deakin Research Online:
This is the authors final peered reviewed (post print) version of the item published as: Adibi,S 2014, A low overhead scaled equalized harmonicbased voice authentication system, Telematics and informatics,
More informationCluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico
Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from
More informationCommandinduced Tracking Jitter Study I D. Clark November 24, 2009
Commandinduced Tracking Jitter Study I D. Clark November 24, 2009 Introduction Reports of excessive tracking jitter on the MMT elevation axis have lately been theorized to be caused by the input command
More informationL15: statistical clustering
Similarity measures Criterion functions Cluster validity Flat clustering algorithms kmeans ISODATA L15: statistical clustering Hierarchical clustering algorithms Divisive Agglomerative CSCE 666 Pattern
More informationAlternative Biometric as Method of Information Security of Healthcare Systems
Alternative Biometric as Method of Information Security of Healthcare Systems Ekaterina Andreeva SaintPetersburg State University of Aerospace Instrumentation SaintPetersburg, Russia eandreeva89@gmail.com
More informationA Learning Based Method for SuperResolution of Low Resolution Images
A Learning Based Method for SuperResolution of Low Resolution Images Emre Ugur June 1, 2004 emre.ugur@ceng.metu.edu.tr Abstract The main objective of this project is the study of a learning based method
More informationSpot me if you can: Uncovering spoken phrases in encrypted VoIP conversations
Spot me if you can: Uncovering spoken phrases in encrypted VoIP conversations C. Wright, L. Ballard, S. Coull, F. Monrose, G. Masson Talk held by Goran Doychev Selected Topics in Information Security and
More informationDigital Modulation. David Tipper. Department of Information Science and Telecommunications University of Pittsburgh. Typical Communication System
Digital Modulation David Tipper Associate Professor Department of Information Science and Telecommunications University of Pittsburgh http://www.tele.pitt.edu/tipper.html Typical Communication System Source
More information4.3 AnalogtoDigital Conversion
4.3 AnalogtoDigital Conversion overview including timing considerations block diagram of a device using a DAC and comparator example of a digitized spectrum number of data points required to describe
More informationIMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH
IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH Kalinka Mihaylova Kaloyanova St. Kliment Ohridski University of Sofia, Faculty of Mathematics and Informatics Sofia 1164, Bulgaria
More informationInvestigating Clinical Care Pathways Correlated with Outcomes
Investigating Clinical Care Pathways Correlated with Outcomes Geetika T. Lakshmanan, Szabolcs Rozsnyai, Fei Wang IBM T. J. Watson Research Center, NY, USA August 2013 Outline Care Pathways Typical Challenges
More informationFrequency Domain Characterization of Signals. Yao Wang Polytechnic University, Brooklyn, NY11201 http: //eeweb.poly.edu/~yao
Frequency Domain Characterization of Signals Yao Wang Polytechnic University, Brooklyn, NY1121 http: //eeweb.poly.edu/~yao Signal Representation What is a signal Timedomain description Waveform representation
More informationPrivate Record Linkage with Bloom Filters
To appear in: Proceedings of Statistics Canada Symposium 2010 Social Statistics: The Interplay among Censuses, Surveys and Administrative Data Private Record Linkage with Bloom Filters Rainer Schnell,
More informationDIGITALTOANALOGUE AND ANALOGUETODIGITAL CONVERSION
DIGITALTOANALOGUE AND ANALOGUETODIGITAL CONVERSION Introduction The outputs from sensors and communications receivers are analogue signals that have continuously varying amplitudes. In many systems
More informationData Mining and Visualization
Data Mining and Visualization Jeremy Walton NAG Ltd, Oxford Overview Data mining components Functionality Example application Quality control Visualization Use of 3D Example application Market research
More informationEnavigation, from sensors to ship behaviour analysis
Enavigation, from sensors to ship behaviour analysis Laurent ETIENNE, Loïc SALMON French Naval Academy Research Institute Geographic Information Systems Group laurent.etienne@ecolenavale.fr loic.salmon@ecolenavale.fr
More informationBasic Acoustics and Acoustic Filters
Basic CHAPTER Acoustics and Acoustic Filters 1 3 Basic Acoustics and Acoustic Filters 1.1 The sensation of sound Several types of events in the world produce the sensation of sound. Examples include doors
More informationUSING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS
USING SPECTRAL RADIUS RATIO FOR NODE DEGREE TO ANALYZE THE EVOLUTION OF SCALE FREE NETWORKS AND SMALLWORLD NETWORKS Natarajan Meghanathan Jackson State University, 1400 Lynch St, Jackson, MS, USA natarajan.meghanathan@jsums.edu
More information