BCB 713 Module Spring 2011

Size: px

Start display at page:

Download "BCB 713 Module Spring 2011"

Posy Mason
7 years ago
Views:

1 Sequential Pattern Mining COMP Seminar BCB 713 Module Spring 2011 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Sequential Pattern Mining Why sequential pattern mining? GSP algorithm FreeSpan and PrefixSpan Boarder Collapsing Constraints and extensions 2

2 Sequence Databases and Sequential Pattern Analysis (Temporal) order is important in many situations Time-series databases and sequence databases Frequent patterns (frequent) sequential patterns Applications of sequential pattern mining Customer shopping sequences: First buy computer, then CD-ROM, and then digital camera, within 3 months. Medical treatment, natural disasters (e.g., earthquakes), science & engineering processes, stocks and markets, telephone calling patterns, Weblog click streams, DNA sequences and gene structures 3 What Is Sequential Pattern Mining? 4 Given a set of sequences, find the complete set of frequent subsequences A sequence database SID sequence 10 <a(abc)(ac)d(cf)> 20 <(ad)c(bc)(ae)> A sequence : < (ef) (ab) (df) c b > An element may contain a set of items. Items within an element are unordered and we list them alphabetically. 30 <(ef)(ab)(df)cb> b <a(bc)dc> is a subsequence 40 <eg(af)cbc> of <a(abc)(ac)d(cf)> Given support tthresholdh min_sup =2, <(ab)c> is a sequential pattern

3 Challenges on Sequential Pattern Mining A huge number of possible sequential patterns are hidden in databases A mining algorithm should Find the complete set of patterns satisfying the minimum support (frequency) threshold Be highly efficient, scalable, involving only a small number of database scans Be able to incorporate various kinds of user- specific constraints 5 A Basic Property of Sequential Patterns: Apriori A basic property: Apriori (Agrawal & Sirkant 94) If a sequence S is not frequent Then none of the super-sequences of fsi is frequent E.g, <hb> is infrequent so do <hab> and <(ah)b> Seq. ID Sequence <(bd)cb(ac)> <(bf)(ce)b(fg)> ) ( p <(ah)(bf)abf> <(be)(ce)d> <a(bd)bcb(ade)> Given support threshold min_sup =2 6

4 Basic Algorithm : Breadth First Search (GSP) L=1 While (Result L!= NULL) Candidate Generate Prune Test L=L+1 7 Finding Length-1 Sequential Patterns 8 Cand Sup Initial candidates: all singleton sequences <a>, , <c>, <d>, <e>, <f>, <g>, <h> <a> 3 Scan database once, count support for 5 candidates <c> 4 <d> 3 min_sup =2 <e> 3 Seq. ID Sequence 10 <(bd)cb(ac)> b( ) <f> 2 20 <(bf)(ce)b(fg)> <g> 1 30 <(ah)(bf)abf> <h> <(be)(ce)d> <a(bd)bcb(ade)>

5 The Mining Process 5 th scan: 1 cand. 1 length-5 seq. <(bd)cba> Cand. cannot pass pat. sup. threshold 4 th scan: 8 cand. 6 length-4 seq. pat. <abba> <(bd)bc> Cand. not in DB at all 3 rd scan: 46 cand. 19 length-3 seq. <abb> <aab> <aba> <baa> <bab> pat. 20 cand. not in DB at all 2 nd scan: 51 cand. 19 length-2 seq. pat. 10 cand. not in DB at all <aa> <ab> <af> <ba> <bb> <ff> <(ab)> <(ef)> 1 st scan: 8 cand. 6 length-1 seq. pat. <a> <c> <d> <e> <f> <g> <h> Seq. ID Sequence min_sup =2 10 <(bd)cb(ac)> 20 <(bf)(ce)b(fg)> 30 <(ah)(bf)abf> <(be)(ce)d> <a(bd)bcb(ade)> 9 Generating Length-2 Candidates 51 length-2 Candidates <a> <c> <d> <e> <f> <a> <(ab)> <(ac)> <(ad)> <(ae)> <(af)> <(bc)> <(bd)> <(be)> <(bf)> <c> <(cd)> <(ce)> <(cf)> <d> <(de)> <(df)> <e> <f> 10 <a> <c> <d> <e> <f> <a> <aa> <ab> <ac> <ad> <ae> <af> <ba> <bb> <bc> <bd> <be> <bf> <c> <ca> <cb> <cc> <cd> <ce> <cf> <d> <da> <db> <dc> <dd> <de> <df> <e> <ea> <eb> <ec> <ed> <ee> <ef> <f> <fa> <fb> <fc> <fd> <fe> <ff> <(ef)> Without Apriori property, 8*8+8*7/2=92 candidates Apriori prunes 44.57% candidates

6 Generating Length-4 Candidates Frequent Candidates Candidates 3-Sequences (after join) (After pruning) < 1 2 3> < > < > < 1 2 4> < > < 1 3 4> < > < > < > < 234> 11 Pattern Growth (prefixspan) Prefix and Suffix (Projection) <a>, <aa>, <a(ab)> and <a(abc)> are prefixes of sequence <a(abc)(ac)d(cf)> Given sequence <a(abc)(ac)d(cf)> ) ( ) Prefix <a> <aa> Suffix (Prefix-Based Projection) <(abc)(ac)d(cf)> <(_bc)(ac)d(cf)> <ab> <(_c)(ac)d(cf)> )d( 12

7 Example 13 Sequence_id Sequence 10 <a(abc)(ac)d(cf)> An Example 20 <(ad)c(bc)(ae)> ( min_sup=2): 30 <(ef)(ab)(df)cb> b> 40 <eg(af)cbc> Prefix Sequential Patterns <a> <a>,<aa>,<ab><a(bc)>,<a(bc)a>,<aba>,<abc>,<(ab)>,<(ab)c>,<(a b)d>,<(ab)f>,<(ab)dc>,<ac>,<aca>,<acb>,<acc>,<ad>,<adc>,<af> , <ba>, <bc>, <(bc)>, <(bc)a>, <bd>, <bdc>,<bf> <c> <c>, <ca>, <cb>, <cc> <d> <d>,<db>,<dc>, <dcb> <> <e> <e>,<ea>,<eab>,<eac>,<eacb>,<eb>,<ebc>,<ec>,<ecb>,<ef>,<efb >< >< >< >< f>< fb >,<efc>,<efcb> <f> <f>,<fb>,<fbc>, <fc>, <fcb> PrefixSpan (the example to be continued) Step1: Find length-1 sequential patterns; <a>:4, :4, <c>:4, <d>:3, <e>:3, <f>:3 Step2: Divide search space; six subsets according to the six prefixes; support pattern Step3: Find subsets of sequential patterns; By constructing corresponding projected databases and mine each recursively. 14

8 Example to be continued Sequence_id Sequence Projected(suffix) databases 10 <a(abc)(ac)d(cf)> <a(abc)(ac)d(cf)> 20 <(ad)c(bc)(ae)> <(ad)c(bc)(ae)> 30 <(ef)(ab)(df)cb> <(ef)(ab)(df)cb> 40 <eg(af)cbc> <eg(af)cbc> Prefix Projected(suffix) databases Sequential Patterns <a> <(abc)(ac)d(cf)>, <a>,<aa>,<ab><a(bc)>,<a(bc)a>, <aa> <ab><a(bc)> <a(bc)a> <(_d)c(bc)(ae)>, <aba>,<abc>,<(ab)>,<(ab)c>,<(ab <(_b)(df)cb>, )d>,<(ab)f>,<(ab)dc>,<ac>,<aca>,<acb>,<acc>,<ad>,<adc>,<af>,,,, <(_f)cbc> 15 Example 16 Find sequential patterns having prefix <a>: 1. Scan sequence database S once. Sequences in S containing <a> are projected w.r.t <a> to form the <a>projected database. 2. Scan <a>-projected database once, get six length-2 sequential patterns having prefix <a> : <a>:2, :4, <(_b)>:2, <c>:4, <d>:2, <f>:2 <aa>:2, <ab>:4, <(ab)>:2, <ac>:4, <ad>:2, <af>:2 3. Recursively, all sequential patterns having prefix <a> can be further partitioned into 6 subsets. Construct respective projected databases and mine each. e.g. <aa>-projected database has two sequences : <(_bc)(ac)d(cf)> and <(_e)>.

9 PrefixSpan Algorithm Main Idea: Use frequent prefixes to divide the search space and to project sequence databases. only search the relevant sequences. PrefixSpan(, i, S ) 1. Scan S once, find the set of frequent items b such that b can be assembled to the last element of to form a sequential pattern; or can be appended to to form a sequential pattern. 2. For each frequent item b, appended it to to form a sequential pattern, and output ; 3. For each, construct -projected database S, and call PrefixSpan(, i+1,s ). 17 Approximate match Compatibility Matrix When you observe d1 Spread count as d1: 90%, d2: 5%, d3: 5% 18

10 Match The degree to which pattern P is retained/reflected in S M(P,S) = P(P S)= C(p,s) when when l S =l P 19 M(P,S) = max over all possible when l S >l P Example P S M d1d1 d1d3 0.9*0 d1d2 d1d2 0.9*0.8 d1d2 d1d3 0.9*0.05 d1d2 d2d3 01* d1d2 d1d2d3 0.9*0.8 Calculate Max over all Dynamic Programming M(p 1 p 2..p i, s 1s 2 s j j) )= Max of O(l P *l S ) M(p 1 p 2..p i-1, s 1 s 2 s j-1 ) * C(p i,s j ) M(p 1 p 2..p i, s 1 s 2 s j-1 ) When compatibility Matrix is sparse O(l S ) 20

11 Match in D Average over all sequences in D 21 Spread of match If compatibility matrix is identity matrix Match = support 22

12 Anti-Monotone The match of a pattern P in a symbol sequence S is less than or equal to the match of any subpattern of fpi in S The match of a pattern P in a sequence database D is less than or equal to the match of any subpattern of P in D Can use any support based algorithm More patterns match so require efficient solution Sample based algorithms Border collapsing of ambiguous patterns 23 Chernoff Bound Given sample size=n, range R, with probability 1- true value: = sqrt([r 2 ln(1/ )]/2n) Distribution ib ti free More conservative Sample size : fit in memory Restricted spread : For pattern P= p 1 p 2..p L R=min (match[p i ]) for all 1 i L Frequent Patterns min_match + min_match - Infrequent patterns 24

13 Algorithm Scan DB: O(N*min (L s *m, L s +m 2 )) Find the match of each individual symbol Take a random sample of sequences Identify borders that embrace the set of ambiguous patterns O(m Lp * S * Lp * n) Min_match existing methods for association rule mining Locate the border of frequent patterns in the entire DB via border collapsing 25 Border Collapsing If memory can not hold the counters of all ambiguous patterns Probe-and-collapse : binary search Probe patterns with highest collapsing power until memory is filled If memory can hold all patterns up to the 1/x layer the space of ambiguous patterns can be narrowed to at least 1/x of the original one where x is a power of 2 If it takes a level-wise search y scans of the DB, only O(log x y) scans are necessary when the border collapsing technique is employed 26

14 Studies on Sequential Pattern Mining Concept introduction and an initial Apriori- like algorithm [AgSr95] GSP An Apriori-based, influential mining method [SrAg96] Mining sequential patterns with constraints [GRS99] Mining long sequential pattern [JWY02] 28

15 Periodic Pattern Full periodic pattern ABC ABC ABC Partial periodic pattern ABCADCACCABC ADC ACC Pattern hierarchy ABC ABC ABC DE DE DE DE ABC ABC ABC DE DE DE DE ABC ABC ABC DE DE DE DE 29 Periodic Pattern Recent Achievements Partial Periodic Pattern Asynchronous Periodic Pattern Meta Pattern InfoMiner/InfoMiner+/STAMP 30

16 Clustering Sequential Data CLUSEQ ApproxMAP 31

Visa Smart Debit/Credit Certificate Authority Public Keys

CHIP AND NEW TECHNOLOGIES Visa Smart Debit/Credit Certificate Authority Public Keys Overview The EMV standard calls for the use of Public Key technology for offline authentication, for aspects of online