Scalable Mining of Large Disk-based Graph Databases

Transcription

1 Sclle Mining of Lrge Disk-sed Grph Dtses Chen Wng Wei Wng Jin Pei Yongti Zhu Bile Shi Fudn University, Chin, {chenwng, weiwng1, 2465, Stte University of New York t Bufflo, USA & Simon Frser University, Cnd, [email protected] ABSTRACT Mining frequent structurl ptterns from grph dtses is n interesting prolem with rod pplictions. Most of the previous studies focus on pruning unfruitful serch suspces effectively, ut few of them ddress the mining on lrge, disk-sed dtses. As mny grph dtses in pplictions cnnot e held into min memory, sclle mining of lrge, disk-sed grph dtses remins chllenging prolem. In this pper, we develop n effective inde structure, ADI (for djcency inde), to support mining vrious grph ptterns over lrge dtses tht cnnot e held into min memory. The inde is simple nd efficient to uild. Moreover, the new inde structure cn e esily dopted in vrious eisting grph pttern mining lgorithms. As n emple, we dpt the well-known gspn lgorithm y using the ADI structure. The eperimentl results show tht the new inde structure enles the sclle grph pttern mining over lrge dtses. In one set of the eperiments, the new disk-sed method cn mine grph dtses with one million grphs, while the originl gspn lgorithm cn only hndle dtses of up to 3 thousnd grphs. Moreover, our new method is fster thn gspn when oth cn run in min memory. Ctegories nd Suject Descriptors: H.2.8 [Dtse Applictions]: Dt Mining Generl Terms: Algorithms, Performnces. Keywords: Grph mining, inde, grph dtse, frequent grph pttern. 1. INTRODUCTION Mining frequent grph ptterns is n interesting reserch prolem with rod pplictions, including mining struc- This reserch is supported in prt y NSF grnt IIS- 381 nd Ntionl Nturl Science Foundtion of Chin (No. 6338). All opinions, findings, conclusions nd recommendtions in this pper re those of the uthors nd do not necessrily reflect the views of the funding gencies. Permission to mke digitl or hrd copies of ll or prt of this work for personl or clssroom use is grnted without fee provided tht copies re not mde or distriuted for profit or commercil dvntge nd tht copies er this notice nd the full cittion on the first pge. To copy otherwise, to repulish, to post on servers or to redistriute to lists, requires prior specific permission nd/or fee. KDD 4, August 22 25, 24, Settle, Wshington, USA. Copyright 24 ACM /4/8...$5.. turl ptterns from chemicl compound dtses, pln dtses, XML documents, we logs, cittion networks, nd so forth. Severl efficient lgorithms hve een proposed in the previous studies [2, 5, 6, 8, 11, 9], rnging from mining grph ptterns, with nd without constrints, to mining closed grph ptterns. Most of the eisting methods ssume implicitly or eplicitly tht the dtses re not very lrge, nd the grphs in the dtse re reltively simple. Tht is, either the dtses or the mjor prt of them cn fit into min memory, nd the numer of possile lels in the grphs [6] is smll. For emple, [11] reports the performnce of gspn, n efficient frequent grph pttern mining lgorithm, on dt sets of size up to 32 KB, using computer with 448 MB min memory. Clerly, the grph dtse nd the projected dtses cn e esily ccommodted into min memory. Under the lrge min memory ssumption, the computtion is CPU-ounded insted of I/O-ounded. Then, the lgorithms focus on effective heuristics to prune the serch spce. Few of them ddress the concern of hndling lrge grph dtses tht cnnot e held in min memory. While the previous studies hve mde ecellent progress in mining grph dtses of moderte size, mining lrge, disk-sed grph dtses remins chllenging prolem. When mining grph dtse tht cnnot fit into min memory, the lgorithms hve to scn the dtse nd nvigte the grphs repetedly. The computtion ecomes I/Oounded. For emple, we otin the eecutle of gspn from the uthors nd test its sclility. In one of our eperiments 1, we increse the numer of grphs in the dtse to test the sclility of gspn on the dtse size. gspn cn only hndle up to 3 thousnd grphs. In nother eperiment, we increse the numer of possile lels in grphs. We oserve tht the runtime of gspn increses eponentilly. It finishes dt set of 3 thousnd grphs with 636 seconds when there re only 1 possile lels, ut needs 15 hours for dt set with the sme size ut the numer of possile lels is 45! This result is consistent with the results reported in [11]. Are there ny rel-life pplictions tht need to mine lrge grph dtses? The nswer is yes. For emple, in dt integrtion of XML documents or mining semntic we, it is often required to find the common sustructures from huge collection of XML documents. It is esy to see pplictions with collections of millions of XML documents. There re 1 Detils will e provided in Section 6

2 hundreds of even thousnds of different lels. As nother emple, chemicl structures cn e modeled s grphs. A chemicl dtse for drug development cn contin millions of different chemicl structures, nd the numer of different lels in the grphs cn esily go to up to 1. These lrge dtses re disk-sed nd often cnnot e held into min memory. Why is mining lrge disk-sed grph dtses so chllenging? In most of the previous studies, the mjor dt structures re designed for eing held in min memory. For emple, the djcency-list or djcency-mtri representtions re often used to represent grphs. Moreover, most of the previous methods re sed on efficient rndom ccesses to elements (e.g., edges nd their djcent edges) in grphs. However, if the djcency-list or djcency-mtri representtions cnnot e held in min memory, the rndom ccesses to them ecome very epensive. For disk-sed dt, without ny inde, rndom ccesses cn e etremely costly. Cn we mke mining lrge, disk-sed grph dtses fesile nd sclle? This is the motivtion of our study. Since the ottleneck is the rndom ccesses to the lrge disk-sed grph dtses, nturl ide is to inde the grph dtses properly. Designing effective nd efficient inde structures is one of the most invlule eercises in dtse reserch. A good inde structure cn support generl ctegory of dt ccess opertions. Prticulrly, good inde should e efficient nd sclle in construction nd mintennce, nd fst for dt ccess. Insted of inventing new lgorithms to mine lrge, disksed grph ptterns, cn we devise n efficient inde structure for grph dtses so tht mining vrious grph ptterns cn e conducted sclly? Moreover, the inde structure should e esy to e dopted in vrious eisting methods with minor dpttions. Stimulted y the ove thinking, in this pper, we study the prolem of efficient inde for sclle mining of lrge, disk-sed grph dtses, nd mke the following contriutions. By nlyzing the frequent grph pttern mining prolem nd the typicl grph pttern mining lgorithms (tking gspn s n emple), we identify severl ottleneck dt ccess opertions in mining lrge, disksed grph dtses. We propose ADI (for djcency inde), n effective inde structure for grphs. We show tht the mjor opertions in grph mining cn e fcilitted efficiently y n ADI structure. The construction lgorithm of ADI structure is presented. We dpt the gspn lgorithm y using the ADI structure on mining lrge, disk-sed grph dtses, nd chieve lgorithm. We show tht outperforms gspn in mining comple grph dtses nd cn mine much lrger dtses thn gspn. A systemtic performnce study is reported to verify our design. The results show tht our new inde structure nd lgorithm re sclle on lrge dt sets. The reminder of the pper is orgnized s follows. We define the prolem of frequent grph pttern mining in Section 2. The ide of minimum DFS code nd lgorithm gspn z y z z v2 v v1 y v3 z v3 v2 v1 y v () Grph () Sugrph (c) DFS-tree (d) DFS-tree G G T 1 T 2 Figure 1: Sugrph nd DFS codes re reviewed in Section 3, nd the mjor dt ccess opertions in grph mining re lso identified. The ADI structure is developed in Section 4. The efficient lgorithm for mining lrge, disk-sed grph dtses using ADI is presented in Section 5. The eperimentl results re reported in Section 6. The relted work is discussed in Section 7. Section 8 concludes the pper. 2. PROBLEM DEFINITION In this pper, we focus on undirected leled simple grphs. A leled grph is 4-tuple G = (V, E, L, l), where V is set of vertices, E V V is set of edges, L is set of lels, nd l : V E L is leling function tht ssigns lel to n edge or verte. We denote the verte set nd the edge set of grph G y V (G) nd E(G), respectively. A grph G is clled connected if for ny vertices u, v V (G), there eist vertices w 1,..., w n V (G) such tht {(u, w 1), (w 1, w 2),..., (w n 1, w n), (w n, v)} E(G). Frequent ptterns in grphs re defined sed on sugrph isomorphism. Definition 1 (Sugrph isomorphism). Given grphs G = (V, E, L, l) nd G = (V, E, L, l ). An injective function f : V V is clled sugrph isomorphism from G to G if (1) for ny verte u V, f(u) V nd l (u) = l(f(u)); nd (2) for ny edge (u, v) E, (f(u), f(v)) E nd l (u, v) = l(f(u), f(v)). If there eists sugrph isomorphism from G to G, then G is clled sugrph of G nd G is clled supergrph of G, denoted s G G. For emple, the grph G in Figure 1() is sugrph of G in Figure 1(). A grph dtse is set of tuples (gid, G), where gid is grph identity nd G is grph. Given grph dtse GDB, the support of grph G in GDB, denoted s sup(g ) for short, is the numer of grphs in the dtse tht re supergrphs of G, i.e., {(gid, G) GDB G G}. For support threshold min sup ( min sup GDB ), grph G is clled frequent grph pttern if sup(g ) min sup. In mny pplictions, users re only interested in the frequent recurring components of grphs. Thus, we put constrint on the grph ptterns: we only find the frequent grph ptterns tht re connected. Prolem definition. Given grph dtse GDB nd support threshold min sup. The prolem of mining frequent connected grph ptterns is to find the complete set of connected grphs tht re frequent in GDB.

3 3. MINIMUM DFS CODE AND GSPAN In [11], Yn nd Hn developed the leicogrphic ordering technique to fcilitte the grph pttern mining. They lso propose n efficient lgorithm, gspn, one of the most efficient grph pttern mining lgorithms so fr. In this section, we review the essentil ides of gspn, nd point out the ottlenecks in the grph pttern mining from lrge disk-sed dtses. 3.1 Minimum DFS Code In order to enumerte ll frequent grph ptterns efficiently, we wnt to identify liner order on representtion of ll grph ptterns such tht if two grphs re in identicl representtion, then they re isomorphic. Moreover, ll the (possile) grph ptterns cn e enumerted in the order without ny redundncy. The depth-first serch tree (DFS-tree for short) [3] is populrly used for nvigting connected grphs. Thus, it is nturl to encode the edges nd vertices in grph sed on its DFS-tree. All the vertices in G cn e encoded in the pre-order of T. However, the DFS-tree is generlly not unique for grph. Tht is, there cn e multiple DFS-trees corresponding to given grph. For emple, Figures 1(c) nd 1(d) show two DFS-trees of the grph G in Figure 1(). The thick edges in Figures 1(c) nd 1(d) re those in the DFS-trees, nd re clled forwrd edges, while the thin edges re those not in the DFS-trees, nd re clled ckwrd edges. The vertices in the grph re encoded v to v 3 ccording to the pre-order of the corresponding DFS-trees. To solve the uniqueness prolem, minimum DFS code nottion is proposed in [11]. For ny connected grph G, let T e DFS-tree of G. Then, n edge is lwys listed s (v i, v j ) such tht i < j. A liner order on the edges in G cn e defined s follows. Given edges e = (v i, v j) nd e = (v i, v j ). e e if (1) when oth e nd e re forwrd edges (i.e., in DFS-tree T ), j < j or (i > i j = j ); (2) when oth e nd e re ckwrd edges (i.e., edges not in DFS-tree T ), i < i or (i = i j < j ); (3) when e is forwrd edge nd e is ckwrd edge, j i ; or (4) when e is ckwrd edge nd e is forwrd edge, i < j. For grph G nd DFS-tree T, list of ll edges in E(G) in order is clled the DFS code of G with respect to T, denoted s code(g, T ). For emple, the DFS code with respect to the DFS-tree T 1 in Figure 1(c) is code(g, T 1) = (v, v 1,,, )-(v 1, v 2,,, z)-(v 2, v, z,, )-(v 1, v 3,,, y), where n edge (v i, v j ) is written s (v i, v j, l(v i ), l(v i, v j ), l(v j )), i.e., the lels re included. Similrly, the DFS code with respect to the DFS-tree T 2 in Figure 1(d) is code(g, T 2) = (v, v 1, y,, )-(v 1, v 2,,, )-(v 2, v 3,,, z)- (v 3, v 1, z,, ). Suppose there is liner order over the lel set L. Then, for DFS-trees T 1 nd T 2 on the sme grph G, their DFS codes cn e compred leiclly ccording to the lels of the edges. For emple, we hve code(g, T 1) < code(g, T 2) in Figures 1(c) nd 1(d). The leiclly minimum DFS code is selected s the representtion of the grph, denoted s min(g). In our emple in Figure 1, min(g) = code(g, T 1). Minimum DFS code hs nice property: two grphs G nd G re isomorphic if nd only if min(g) = min(g ). Moreover, with the minimum DFS code of grphs, the pro- Input: DFS code s, grph dtse GDB nd min sup Output: the frequent grph ptterns Method: if s is not minimum DFS code then return; output s s pttern if s is frequent in GDB; let C = ; scn GDB once, find every edge e such tht e cn e conctented to s to form DFS code s e nd s e is frequent; C = C {s e}; sort the DFS codes in C in leicogrphic order; for ech s e C in leicogrphic order do cll gspn(s e, GDB, min sup); return; Figure 2: Algorithm gspn. lem of mining frequent grph ptterns is reduced to mining frequent minimum DFS codes, which re sequences, with some constrints tht preserve the connectivity of the grph ptterns. 3.2 Algorithm gspn Bsed on the minimum DFS codes of grphs, depthfirst serch, pttern-growth lgorithm, gspn, is developed in [11], s shown in Figure 2. The centrl ide is to conduct depth-first serch of minimum DFS codes of possile grph ptterns, nd otin longer DFS codes of lrger grph ptterns y ttching new edges to the end of the minimum DFS code of the eisting grph pttern. The nti-monotonicity of frequent grph ptterns, i.e., ny super pttern of n infrequent grph pttern cnnot e frequent, is used to prune. Compring to the previous methods on grph pttern mining, gspn is efficient, since gspn employs the smrt ide of minimum DFS codes of grph ptterns tht fcilittes the isomorphism test nd pttern enumertion. Moreover, gspn inherits the depth-first serch, pttern-growth methodology to void ny cndidte-genertion-nd-test. As reported in [11], the dvntges of gspn re verified y the eperimentl results on oth rel dt sets nd synthetic dt sets. 3.3 Bottlenecks in Mining Disk-sed Grph Dtses Algorithm gspn is efficient when the dtse cn e held into min memory. For emple, in [11], gspn is sclle for dtses of size up to 32 KB using computer with 448 MB min memory. However, it my encounter difficulties when mining lrge dtses. The mjor overhed is tht gspn hs to rndomly ccess elements (e.g., edges nd vertices) in the grph dtse s well s the projections of the grph dtse mny times. For dtses tht cnnot e held into min memory, the mining ecomes I/O ounded nd thus is costly. Rndom ccesses to elements in grph dtses nd checking the isomorphism re not unique to gspn. Insted, such opertions re etensive in mny grph pttern mining lgorithms, such s FSG [6] (nother efficient frequent grph pttern mining lgorithm) nd CloseGrph [9] (n efficient lgorithm for mining frequent closed grph ptterns). In mining frequent grph ptterns, the mjor dt ccess opertions re s follows.

4 OP1: Edge support checking. Find the support of n edge (l u, l e, l v ), where l u nd l v re the lels of vertices nd l e is the lel of the edge, respectively; OP2: Edge-host grph checking. For n edge e = (l u, l e, l v ), find the grphs in the dtse where e ppers; OP3: Adjcent edge checking. For n edge e = (l u, l e, l v), find the djcent edges of e in the grphs where e ppers, so tht the djcent edges cn e used to epnd the current grph pttern to lrger ones. Ech of the ove opertions my hppen mny times during the mining of frequent grph ptterns. Without n pproprite inde, ech of the ove opertions my hve to scn the grph dtse or its projections. If the dtse nd its projections cnnot fit into min memory, the scnning nd checking cn e very costly. Cn we devise n inde structure so tht the relted informtion cn e kept nd ll the ove opertions cn e chieved using the inde only, nd thus without scnning the grph dtse nd checking the grphs? This motivtes the design of the ADI structure. 4. THE ADI STRUCTURE In this section we will devise n effective dt structure, ADI (for djcency inde), to fcilitte the sclle mining of frequent grph ptterns from disk-sed grph dtses. 4.1 Dt Structure The ADI inde structure is three-level inde for edges, grph-ids nd djcency informtion. An emple is shown in Figure 3, where two grphs, G 1 nd G 2, re indeed Edge Tle There cn e mny edges in grph dtse. The edges re often retrieved y the lels during the grph pttern mining, such s in the opertions identified in Section 3.3. Therefore, the edges re indeed y their lels in the ADI structure. In ADI, n edge e = (u, v) is recorded s tuple (l(u), l(u, v), l(v)) in the edge tle, nd is indeed y the lels of the vertices, i.e., l(u) nd l(v), nd the lel of the edge itself, i.e., l(u, v). Ech edge ppers only once in the edge tle, no mtter how mny times it ppers in the grphs. For emple, in Figure 3, edge (A, d, C) ppers once in grph G 1 nd twice in grph G 2. However, there is only one entry for the edge in the edge tle in the ADI structure. All edges in the edge tle in the ADI structure re sorted. When the edge tle is stored on disk, B+-tree is uilt on the edges. When prt of the edge tle is loded into min memory, it is orgnized s sorted list. Thus, inry serch cn e conducted Linked Lists of Grph-ids For ech edge e, the identities of the grphs tht contin e form linked list of grph-ids. Grph-id G i is in the list of edge e if nd only if there eists t lest one instnce of e in G i. For emple, in Figure 3, oth G 1 nd G 2 pper in the list of edge (A, d, C), since the edge ppers in G 1 once nd in G 2 twice. Plese note tht the identity of grph G i B 2 Edges (A,, B) (A, d, C) (B,, D) (B, c, C) (B, d, D) (C, d, D) 1 A 3 D d 4 C d G1 G2 Grph ids (on disk) Adjcency (on disk) G1 G2 G1 G2 G1 G2 G2 G1 d 4 C c 5 B A 1 d C B 2 6 d 3 D Block Block Figure 3: An ADI structure. ppers in the linked list of edge e only once if e ppers in G i, no mtter how mny times edge e ppers in G i. A list of grph-ids of n edge re stored together. Therefore, given n edge, it is efficient to retrieve ll the identities of grphs tht contin the edge. Every entry in the edge tle is linked to its grph-id linked list. By this linkge, the opertion OP2: edge-host grph checking cn e conducted efficiently. Moreover, to fcilitte opertion OP1: edge support checking, the length of the grph-id linked list, i.e., the support of n edge, is registered in the edge tle Adjcency Informtion The edges in grph re stored s list of the edges encoded. Adjcent edges re linked together y the common vertices, s shown in Figure 3. For emple, in lock 1, ll the vertices hving the sme lel (e.g., 1) re linked together s list. Since ech edge hs two vertices, only two pointers re needed for ech edge. Moreover, ll the edges in grph re physiclly stored in one lock on disk (or on consecutive locks if more spce is needed), so tht the informtion out grph cn e retrieved y reding one or severl consecutive locks from disk. Often, when the grph is not lrge, disk-pge (e.g., of size 4k) cn hold more thn one grph. Encoded edges recording the djcency informtion re linked to the grph-ids tht re further ssocited with the edges in the edge tle. 4.2 Spce Requirement The storge of n ADI structure is fleile. If the grph dtse is smll, then the whole inde cn e held into min memory. On the other hnd, if the grph dtse is lrge nd thus the ADI structure cnnot fit into min

5 memory, some levels cn e stored on disk. The level of djcency informtion is the most detiled nd cn e put on disk. If the min memory is too smll to hold the grphid linked lists, they cn lso e ccommodted on disk. In the etreme cse, even the edge tle cn e held on disk nd B+-tree or hsh inde cn e uilt on the edge tle. Theorem 1 (Spce compleity). For grph dtse GDB = {G 1,..., G n }, the spce compleity is O( n i=1 E(Gi) ). Proof. The spce compleity is determined y the following fcts. (1) The numer of tuples in the edge tle is equl to the numer of distinct edges in the grph dtse, which is ounded y n i=1 E(G i) ; (2) The numer of entries in the grph-id linked lists in the worst cse is the numer of edges in the grph dtse, i.e., n i=1 E(Gi) gin; nd (3) The djcency informtion prt records every edge ectly once. Plese note tht, in mny ppliction, it is resonle to ssume tht the edge tle cn e held into min memory. For emple, suppose we hve 1, distinct verte lels nd 1, distinct edge lels. There cn e up to = different edges, i.e., ll possile comintions of verte nd edge lels. Suppose up to 1% edges re frequent, there re only less thn 5 million different edges, nd thus the edge tle cn e esily held into min memory. In rel pplictions, the grphs re often sprse, tht is, not ll possile comintions of verte nd edge lels pper in the grphs s n edge. Moreover, users re often interested in only those frequent edges. Tht shrinks the edge tle sustntilly. 4.3 Serch Using ADI Now, let us emine how the ADI structure cn fcilitte the mjor dt ccess opertions in grph pttern mining tht re identified in Section 3.3. OP1: Edge support checking Once n ADI structure is constructed, this informtion is registered on the edge tle for every edge. We only need to serch the edge tle, which is either indeed (when the tle is on disk) or cn e serched using inry serch (when the tle is in min memory). In some cses, we my need to count the support of n edge in suset of grphs G G. Then, the linked list of the grph-ids of the edge is serched. There is no need to touch ny record in the djcency informtion prt. Tht is, we do not need to serch ny detil out the edges. Moreover, for counting supports of edges in projected dtses, we cn mintin the support of ech edge in the current projected dtse nd thus we do not even serch the grph-id linked lists. OP2: Edge-host grph checking We only need to serch the edge tle for the specific edge nd follow the link from the edge to the list of grph-ids. There is no need to serch ny detil from the prt of djcency informtion. OP3: Adjcent edge checking Agin, we strt from n entry in the edge tle nd follow the links to find the list of grphs where the edge ppers. Then, only Input: grph dtse GDB nd min sup Output: the ADI structure Method: scn GDB once, find the frequent edges; initilize the edge tle for frequent edges; for ech grph do remove infrequent edges; compute the mininmum DFS code [11]; use the DFS-tree to encode the vertices; store the edges in the grph onto disk nd form the djcency informtion; for ech edge do insert the grph-id to the grph-id list ssocited with the edge; link the grph-id to the relted djcency informtion; end for end for Figure 4: Algorithm of ADI construction. the locks contining the detils of the instnces of the edge re visited, nd there is no need to scn the whole dtse. The verge I/O compleity is O(log n + m + l), where n is the numer of distinct edges in the grph, m is the verge numer of grph-ids in the linked lists of edges, nd l is the verge numer of locks occupied y grph. In mny pplictions, m is orders of mgnitudes smller thn the n, nd l is very smll numer (e.g., 1 or 2). The lgorithms for the ove opertions re simple. Limited y spce, we omit the detils here. As cn e seen, once the ADI structure is constructed, there is no need to scn the dtse for ny of the ove opertions. Tht is, the ADI structure cn support the rndom ccesses nd the mining efficiently. 4.4 Construction of ADI Given grph dtse, the corresponding ADI structure is esy to construct y scnning the dtse only twice. In the first scn, the frequent edges re identified. According to the priori property of frequent grph ptterns, only those frequent edges cn pper in frequent grph ptterns nd thus should e indeed in the ADI structure. After the first scn, the edge tle of frequent edges is initilized. In the second scn, grphs in the dtse re red nd processed one y one. For ech grph, the vertices re encoded ccording to the DFS-tree in the minimum DFS code, s descried in [11] nd Section 3. Only the vertices involved in some frequent edges should e encoded. Then, for ech frequent edge, the grph-id is inserted into the corresponding linked list, nd the djcency informtion is stored. The sketch of the lgorithm is shown in Figure 4. Cost Anlysis There re two mjor costs in the ADI construction: writing the djcency informtion nd updting the linked lists of grph-ids. Since ll edges in grph will reside on disk pge or severl consecutive disk pges, the writing of djcency informtion is sequentil. Thus, the cost of writing djcency informtion is comprle to tht of mking

6 2 B 1 A 3 D d 4 C d 1 A 2 B 3 D 4 C d 3 d 4 d 4 d 1 () The grph nd the djcency-lists 1 A 2 B 3 D 4 C 1 A d 2 B 3 D d 4 C d d () The djcency-mtri Figure 5: The djcency-list nd djcency-mtri representtions of grphs. copy of the originl dtse plus some ookkeeping. Updting the linked lists of grph-ids requires rndom ccesses to the edge tle nd the linked lists. In mny cses, the edge tle cn e held into min memory, ut not the linked list. Therefore, it is importnt to cche the linked lists of grph-ids in uffer. The linked lists cn e cched ccording to the frequency of the corresponding edges. Constructing ADI for lrge, disk-sed grph dtse my not e chep. However, the ADI structure cn e uilt once nd used y the mining mny times. Tht is, we cn uild n ADI structure using very low support threshold, or even set min sup = 1. 2 The inde is stored on disk. Then, the mining in the future cn use the inde directly, s long s the support threshold is no less thn the one tht is used in the ADI structure construction. 4.5 Projected Dtses Using ADI Mny depth-first serch, pttern-growth lgorithms utilize proper projected dtses. During the depth-first serch in grph pttern mining, the grphs contining the current grph pttern P should e collected nd form the P - projected dtse. Then, the further serch of lrger grph ptterns hving P s the prefi of their minimum DFS codes cn e chieved y serching only the P -projected dtse. Interestingly, the projected dtses cn e constructed using ADI structures. A projected dtse cn e stored in the form of n ADI structure. In fct, only the edge tle nd the list of grph-ids should e constructed for new projected dtse nd the djcency informtion residing on disk cn e shred y ll projected dtses. Tht cn sve lot of time nd spce when mining lrge grph dtses tht contin mny grph ptterns, where mny projected dtses my hve to e constructed. 4.6 Why Is ADI Good for Lrge Dtses? In most of the previous methods for grph pttern mining, the djcency-list or djcency-mtri representtions re used to represent grphs. Ech grph is represented y n djcency-mtri or set of djcency-lists. An emple is shown in Figure 5. 2 If min sup = 1, then the ADI structure cn e constructed y scnning the grph dtse only once. We do not need to find frequent edges, since every edge ppering in the grph dtse is frequent. In Figure 5(), the djcency-lists hve 8 nodes nd 8 pointers. It stores the sme informtion s Block 1 in Figure 3, where the lock hs 4 nodes nd 12 pointers. The spce requirements of djcency-lists nd ADI structure re comprle. From the figure, we cn see tht ech edge in grph hs to e stored twice: one instnce for ech verte. (If we wnt to remove this redundncy, the trdeoff is the sustntil increse of cost in finding djcency informtion). In generl, for grph of n edges, the djcency-list representtion needs 2n nodes nd 2n pointers. An ADI structure stores ech edge once, nd use the linkge mong the edges from the sme verte to record the djcency informtion. In generl, for grph of n edges, it needs n nodes nd 3n pointers. Then, wht is the dvntge of ADI structure ginst djcency-list representtion? The key dvntge is tht the ADI structure etrcts the informtion out continments of edges in grphs in the first two levels (i.e., the edge tle nd the linked list of grph-ids). Therefore, in mny opertions, such s the edge support checking nd edge-host grph checking, there is no need to visit the djcency informtion t ll. To the contrst, if the djcency-list representtion is used, every opertion hs to check the linked lists. When the dtse is lrge so tht either the djcency-lists of ll grphs or the djcency informtion in the ADI structure cnnot e ccommodted into min memory, using the first two levels of the ADI structure cn sve mny clls to the djcency informtion, while the djcency-lists of vrious grphs hve to e trnsferred etween the min memory nd the disk mny times. Usully, the djcency-mtri is sprse. The djcencymtri representtion is inefficient in spce nd thus is not used. 5. ALGORITHM ADI-MINE With the help from the ADI structure, how cn we improve the sclility nd efficiency of frequent grph pttern mining? Here, we present pttern-growth lgorithm ADI- Mine, which is n improvement of lgorithm gspn. The lgorithm is shown in Figure 6. If the ADI structure is unville, then the lgorithm scns the grph dtse nd constructs the inde. Otherwise, it just uses the ADI structure on the disk. The frequent edges cn e otined from the edge tle in the ADI structure. Ech frequent edge is one of the smllest frequent grph ptterns nd thus should e output. Then, the frequent edges should e used s the seeds to grow lrger frequent grph ptterns, nd the frequent djcent edges of e should e used in the pttern-growth. An edge e is frequent djcent edge of e if e is n djcent edge of e in t lest min sup grphs. The set of frequent djcent edges cn e retrieved efficiently from the ADI structure since the identities of the grphs contining e re indeed s linked-list, nd the djcent edges re lso indeed in the djcency informtion prt in the ADI structure. The pttern growth is implemented s clls to procedure sugrph-mine. Procedure sugrph-mine tries every frequent djcent edge e (i.e., edges in set F e ) nd checks whether e cn e dded into the current frequent grph pttern G to form lrger pttern G. We use the DFS code to test the redundncy. Only the ptterns G whose DFS code is minimum is output nd further grown. All other ptterns G re either found efore or will e found lter t other

7 Input: grph dtse GDB nd min sup Output: the complete set of frequent grph ptterns Method: construct the ADI structure for the grph dtse if it is not ville; for ech frequent edge e in the edge tle do output e s grph pttern; from the ADI structure, find set F e, the set of frequent djcent edges for e; cll sugrph-mine(e, F e ); end for Procedure sugrph-mine Prmeters: frequent grph pttern G, nd the set of frequent djcent edges F e // output the frequent grph ptterns whose // minimum DFS-codes contin tht of G s prefi Method: for ech edge e in F e do let G e the grph y dding e into G; compute the DFS code of G ; if the DFS code is not minimum, then return; output G s frequent grph pttern; updte the set F e of djcent edges; cll sugrph-mine(g, F e ); end for return; Figure 6: Algorithm. rnches. The correctness of this step is gurnteed y the property of DFS code [11]. Once lrger pttern G is found, the set of djcent edges of the current pttern should e updted, since the djcent edges of the newly inserted edge should lso e considered in the future growth from G. This updte opertion cn e implemented efficiently, since the identities of grphs tht contin n edge e re linked together in the ADI structure, nd the djcency informtion is lso indeed nd linked ccording to the grph-ids. Differences Between nd gspn At high level, the structure s well s the serch strtegies of nd gspn re similr. The criticl difference is on the storge structure for grphs uses ADI structure nd gspn uses djcency-list representtion. In the recursive mining, the criticl opertion is finding the grphs tht contin the current grph pttern (i.e., the test of sugrph isomorphism) nd finding the djcent edges to grow lrger grph ptterns. The current grph pttern is recorded using the lels. Thus, the edges re serched using the lels of the vertices nd tht of the edges. In gspn, the test of sugrph isomorphism is chieved y scnning the current (projected) dtse. Since the grphs re stored in djcency-list representtion, nd one lel my pper more thn once in grph, the serch cn e costly. For emple, in grph G 2 in Figure 3, in order to find n edge (C, d, A), the djcency-list for vertices 4 nd 6 my hve to e serched. If the grph is lrge nd the lels pper multiple times in grph, there my e mny djcency-lists for vertices of the sme lel, nd the djcency-lists re long. Moreover, for lrge grph dtse tht cnnot e held into min memory, the djcency-list representtion of grph hs to e loded into min memory efore the grph cn e serched. In, the grphs re stored in the ADI structure. The edges re indeed y their lels. Then, the grphs tht contin the edges cn e retrieved immeditely. Moreover, ll edges with the sme lels re linked together y the links etween the grph-id nd the instnces. Tht helps the test of sugrph isomorphism sustntilly. Furthermore, using the inde of edges y their lels, only the grphs tht contin the specific edge will e loded into min memory for further sugrph isomorphism test. Irrelevnt grphs cn e filtered out immeditely y the inde. When the dtse is too lrge to fit into min memory, it sves sustntil prt of trnsfers of grphs etween disk nd min memory. 6. EXPERIMENTAL RESULTS In this section, we report systemtic performnce study on the ADI structure nd comprison of gspn nd ADI- Mine on mining oth smll, memory-sed dtses nd lrge, disk-sed dtses. We otin the eecutle of gspn from the uthors. The ADI structure nd lgorithm re implemented using C/C Eperiment Setting All the eperiments re conducted on n IBM NetFinity 51 mchine with n Intel PIII 733MHz CPU, 512M RAM nd 18G hrd disk. The speed of the hrd disk is 1, RPM. The operting system is Redht Linu 9.. We implement synthetic dt genertor following the procedure descried in [6]. The dt genertor tkes five prmeters s follows. D: the totl numer of grphs in the dt set T : the verge numer of edges in grphs I: the verge numer of edges in potentilly frequent grph ptterns (i.e., the frequent kernels) L: the numer of potentilly frequent kernels N: the numer of possile lels Plese refer to [6] for the detils of the dt genertor. For emple, dt set D1kN4I1T 2L2 mens tht the dt set contins 1k grphs; there re 4 possile lels; the verge numer of edges in the frequent kernel grphs is 1; the verge numer of edges in the grphs is 2; nd the numer of potentilly frequent kernels is 2. Herefter in this section, when we sy prmeters, it mens the prmeters for the dt genertor to crete the dt sets. In [11], L is fied to 2. In our eperiments, we lso set L = 2 s the defult vlue, ut will test the sclility of our lgorithm on L s well. Plese note tht, in ll eperiments, the runtime of ADI- Mine includes oth the ADI construction time nd the mining time. 6.2 Mining Min Memory-sed Dtses In this set of eperiments, oth gspn nd run in min memory.

8 6.2.1 Sclility on Minimum Support Threshold We test the sclility of gspn nd on the minimum support threshold. Dt set D1kN3I5T 2L2 is used. The minimum support threshold vries from 4% to 1%. The results re shown in Figure 7(). As cn e seen, oth gspn nd re sclle, ut is out 1 times fster. We discussed the result with Mr. X. Yn, the uthor of gspn. He confirms tht counting frequent edges in gspn is time consuming. On the other hnd, the construction of ADI structure is reltively efficient. When the minimum support threshold is set to 1, i.e., ll edges re indeed, the ADI structure uses pproimtely 57M min memory nd costs 86 seconds in construction Sclility on Dtse Size We test the sclility of gspn nd on the size of dtses. We fi the prmeters N = 3, I = 5, T = 2 nd L = 2, nd vry the numer of grphs in dtse from 5 thousnd to 1 thousnd. The minimum support threshold is set to 1% of the numer of grphs in the dtse. The results re shown in Figure 7(). The construction time of ADI structure is lso plotted in the figure. Both the lgorithms nd the construction of ADI structure re linerly sclle on the size of dtses. is fster. We oserve tht the size of ADI structure is lso sclle. For emple, it uses 28M when the dtse hs 5 thousnd grphs, nd 57M when the dtse hs 1 thousnd grphs. This oservtion concurs with Theorem Effects of Dt Set Prmeters We test the sclility of the two lgorithms on prmeter N the numer of possile lels. We use dt set D1kN2-5I5T 2L2, tht is, the N vlue vries from 2 to 5. The minimum support threshold is fied t 1%. The results re shown in Figure 7(c). Plese note tht the Y -is is in logrithmic scle. We cn oserve tht the runtime of gspn increses eponentilly s N increses. This result is consistent with the result reported in [11]. 3 When there re mny possile lels in the dtse, the serch without inde ecomes drmticlly more costly. Interestingly, oth nd the construction of ADI structure re linerly sclle on N. As discussed efore, the edge tle in ADI structure only indees the unique edges in grph dtse. Serching using the indeed edge tle is efficient. The time compleity of serching n edge y lels is O(log n), where n is the numer of distinct edges in the dtse. This is not ffected y the increse of the possile lels. As epected, the size of the ADI structure is stle, out 57M in this eperiment. We use dt set D1kN3I5T 1-3L2 to test the sclility of the two lgorithms on prmeter T the verge numer of edges in grph. The minimum support threshold is set to 1%. The results re shown in Figure 7(d). As the numer of edges increses, the grph ecomes more comple. The cost of storing nd serching the grph lso increses ccordingly. As shown in the figure, oth lgorithms nd the construction of ADI re linerly sclle. We lso test the effects of other prmeters. The eperimentl results show tht oth gspn nd re not 3 Plese refer to Figures 5() nd 5(c) in the UIUC technicl report version of [11]. sensitive to I the verge numer of edges in potentilly frequent grph ptterns nd L the numer of potentilly frequent kernels. The construction time nd spce cost of ADI structures re lso stle. The reson is tht the effects of those two prmeters on the distriution in the dt sets re minor. Similr oservtions hve een reported y previous studies on mining frequent itemsets nd sequentil ptterns. Limited y spce, we omit the detils here. 6.3 Mining Disk-sed Dtses Now, we report the eperimentl results on mining lrge, disk-sed dtses. In this set of eperiments, we reserve lock of min memory of fied size for ADI structure. When the size is too smll for the ADI-structure, some levels of the ADI structure re ccommodted on disk. On the other hnd, we do not confine the memory usge for gspn Sclility on Dtse Size We test the sclility of oth gspn nd on the size of dtses. We use dt set D1k-1mN3I5T 2L2. The numer of grphs in the dtse is vried from 1 thousnd to 1 million. The min memory lock for ADI structure is limited to 25M. The results re shown in Figure 8(). The construction time of ADI structure is lso plotted. Plese note tht the Y -is is in logrithmic scle. The construction runtime of ADI structure is pproimtely liner on the dtse size. Tht is, the construction of the ADI inde is highly sclle. We lso mesure the size of ADI structure. The results re shown in Figure 8(). We cn oserve tht the size of the ADI structure is liner to the dtse size. In this eperiment, the rtio size of ADI structure in megytes is out.6. When the numer of grphs in thousnds dtse size is 1 million, the size of ADI structure is 61M, which eceeds the min memory size of our mchine. Even in such cse, the construction runtime is still liner. As eplined efore, the construction of ADI structure mkes sequentil scns of the dtse nd conducts sequentil write of the djcency informtion. The overhed of construction of edge tle nd the linked lists of grphids is reltively smll nd thus hs minor effect on the construction time. While gspn cn hndle dtses of only up to 3 thousnd grphs in this eperiment, cn hndle dtses of 1 million grphs. The curve of the runtime of cn e divided into three stges. First, when the dtse hs up to 3 thousnd grphs, the ADI structure cn e fully ccommodted in min memory. is fster thn gspn. Second, when the dtse hs 3 to 6 thousnd grphs, gspn cnnot finish. The ADI structure cnnot e fully held in min memory. Some prt of the djcency informtion is put on disk. We see significnt jump in the runtime curve of etween the dtses of 3 thousnd grphs nd 4 thousnd grphs. Lst, when the dtse hs 8 thousnd or more grphs, even the linked lists of grph-ids cnnot e fully put into min memory. Thus, nother significnt jump in the runtime curve cn e oserved Trdeoff Between Efficiency nd Min Memory Consumption It is interesting to emine the trdeoff etween efficiency nd size of ville min memory. We use dt set

9 Runtime (second) gspn Runtime (second) gspn ADI-construction Runtime (second) gspn ADI-construction Runtime (second) gspn ADI-construction min_sup (%) Numer of grphs (thousnd) N T () sclility on min sup () Sclility on size (c) Sclility on N (d) Sclility on T D1kN3I5T 2L2 D5-1kN3I5T 2L2 D1kN2-5I5T 2L2 D1kN3I5T 1-3L2 min sup = 1% min sup = 1% min sup = 1% Figure 7: The eperimentl results of mining min memory-sed dtses. Runtime (second) gspn ADI-construction Size (M) ADI structure Runtime (s) Numer of grphs (thousnd) Numer of grphs (thousnd) Size of ville min memory (M) () sclility on size () Size of ADI structure (c) Runtime vs. min memory D1k-1mN3I5T 2L2 D1k-1mN3I5T 2L2 D1kN3I5T 2L2 min sup = 1% min sup = 1% min sup = 1% Figure 8: The eperimentl results of mining lrge disk-sed dtses. D1kN3I5T 2L2, set the minimum support threshold to 1%, vry the min memory limit from 1M to 15M for ADI structure, nd mesure the runtime of. The results re shown in Figure 8(c). In this eperiment, the size of ADI structure is 57M. The construction time is 86 seconds. The highest wtermrk of min memory usge for gspn in mining this dt set is 87M. gspn uses 1161 seconds in the mining if it hs sufficient min memory. When the ADI structure cn e completely loded into min memory (57M or lrger), runs fst. Further increse of the ville min memory cnnot reduce the runtime. When the ADI structure cnnot e fully put into min memory, the runtime increses. The more min memory, the fster runs. When the ville min memory is too smll to even hold the linked lists of grph-ids, the runtime of increses sustntilly. However, it still cn finish the mining with 1M min memory limit in 2 hours Numer of Disk Block Reds In ddition to runtime, the efficiency of mining lrge disksed dtses cn lso e mesured y the numer of disk lock red opertions. Figure 9() shows the numer of disk lock reds versus the minimum support threshold. When the support threshold is high (e.g., 9% or up), the numer of frequent edges is smll. The ADI structure cn e held into min memory nd thus the I/O cost is very low. As the support threshold goes down, lrger nd lrger prt of the ADI structure is stored on disk, nd the I/O cost increses. This curve is consistent with the trend in Figure 7(). Figure 9() shows the numer of disk lock reds versus the numer of grphs in the dtse. As the dtse size goes up, the I/O cost increses eponentilly. This eplins the curve of in Figure 8(). We lso test the I/O cost on ville min memory. The result is shown in Figure 9(c), which is consistent with the trend of runtime curve in Figure 8(c) Effects of Other Prmeters We lso test the effects of the other prmeters on the efficiency. We oserve similr trends s in mining memorysed dtses. Limited y spce, we omit the detils here. 6.4 Summry of Eperimentl Results The etensive performnce study clerly shows the following. First, oth gspn nd re sclle when dtse cn e held into min memory. is fster thn gspn. Second, cn mine very lrge grph dtses y ccommodting the ADI structure on disk. The performnce of on mining lrge disk-sed dtses is highly sclle. Third, the size of ADI structure is linerly sclle with respect to the size of dtses. Fourth, we cn control the trdeoff etween the mining efficiency nd the min memory consumption. Lst, is more sclle thn gspn in mining comple grphs the grphs tht hve mny different kinds of lels. 7. RELATED WORK The prolem of finding frequent common structures hs een studied since erly 199s. For emple, [1, 7] study the the prolem of finding common sustructures from chemicl compounds. SUBDUE [4] proposes n pproimte lgorithm to identify some, insted of the complete set of,

10 Numer of locks red min_sup (%) Numer of locks red 2.5e+8 2e+8 1.5e+8 1e+8 5e Numer of grphs (thousnd) Numer of locks red 4e+7 3.5e+7 3e+7 2.5e+7 2e+7 1.5e+7 1e+7 5e Size of ville min memory (M) () # locks vs. support threshold () # locks vs. dtse size (c) # locks vs. min memory size D1kN3I5T 2L2 D1k-1mN3I5T 2L2 D1kN3I5T 2L2 min sup = 1% min sup = 1% Figure 9: The numer of disk locks red in the mining. frequent sustructures. However, these methods do not im t sclle lgorithms for mining lrge grph dtses. The prolem of mining the complete set of frequent grph ptterns is firstly eplored y Inokuchi et l. [5]. An Apriorilike lgorithm AGM is proposed. Kurmochi nd Krypis [6] develop n efficient lgorithm, FSG, for grph pttern mining. The mjor ide is to utilize n effective grph representtion, nd conduct the edge-growth mining insted of verte-growth mining. Both AGM nd FSG dopt redthfirst serch. Recently, Yn nd Hn propose the depth-first serch pproch, gspn [11] for grph mining. They lso investigte the prolem of mining frequent closed grphs [9], which is non-redundnt representtion of frequent grph ptterns. As ltest result, Yn et l. [1] uses frequent grph ptterns to inde grphs. As specil cse of grph mining, tree mining lso receives intensive reserch recently. Zki [12] proposes the first lgorithm for mining frequent tree ptterns. Although there re quite few studies on the efficient mining of frequent grph ptterns, none of them ddresses the prolem of effective inde structure for mining lrge disksed grph dtses. When the dtse is too lrge to fit into min memory, the mining ecomes I/O ounded, nd the pproprite inde structure ecomes very criticl for the sclility. 8. CONCLUSIONS In this pper, we study the prolem of sclle mining of lrge disk-sed grph dtse. The ADI structure, n effective inde structure, is developed. Tking gspn s concrete emple, we propose, n efficient lgorithm dopting the ADI structure, to improve the sclility of the frequent grph mining sustntilly. The structure is generl inde for grph mining. As future work, it is interesting to emine the effect of the inde structure on improving other grph pttern mining methods, such s mining frequent closed grphs nd mining grphs with constrints. Furthermore, devising inde structures to support sclle dt mining on lrge disk-sed dtses is n importnt nd interesting reserch prolem with etensive pplictions nd industril vlues. Acknowledgements We re very grteful to Mr. Xifeng Yn nd Dr. Jiwei Hn for kindly providing us the eecutle of gspn nd nswering our questions promptly. We would like to thnk the nonymous reviewers for their insightful comments, which help to improve the qulity of the pper. 9. REFERENCES [1] D.M. Byd, R. W. Simpson, nd A. P. Johnson. An lgorithm for the multiple common sugrph prolem. J. of Chemicl Informtion & Computer Sci., 32:68 685, [2] C. Borgelt nd M.R. Berthold. Mining moleculr frgments: Finding relevnt sustructures of molecules. In Proc. 22 Int. Conf. Dt Mining (ICDM 2), Meshi TERRSA, Meshi City, Jpn, Dec. 22. [3] Thoms H. Cormen, Chrles E. Leiserson, Ronld L. Rivest, nd Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press nd McGrw-Hill, 22. [4] L. B. Holder, D. J. Cook, nd S. Djoko. Sustructure discovery in the sudue system. In Proc. AAAI 94 Workshop Knowledge Discovery in Dtses (KDD 94), pges , Settle, WA, July [5] A. Inokuchi, T. Wshio, nd H. Motod. An priori-sed lgorithm for mining frequent sustructures from grph dt. In Proc. 2 Europen Symp. Principle of Dt Mining nd Knowledge Discovery (PKDD ), pges 13 23, Lyon, Frnce, Sept. 2. [6] M. Kurmochi nd G. Krypis. Frequent sugrph discovery. In Proc. 21 Int. Conf. Dt Mining (ICDM 1), pges , Sn Jose, CA, Nov. 21. [7] Y. Tkhshi, Y. Stoh, nd S. Sski. Recognition of lrgest common frgment mong vriety of chemicl structures. Anlyticl Sciences, 3:23 38, [8] N. Vnetik, E. Gudes, nd S.E. Shimony. Computing frequent grph ptterns from semistructured dt. In Proc. 22 Int. Conf. Dt Mining (ICDM 2), Meshi TERRSA, Meshi City, Jpn, Dec. 22. [9] X. Yn nd J. Hn. Closegrph: Mining closed frequent grph ptterns. In Proceedings of the 9th ACM SIGKDD Interntionl Conference on Knowledge Discovery nd Dt Mining (KDD 3), Wshington, D.C, 23. [1] X. Yn, P.S. Yu, nd J. Hn. Grph indeing: A frequent structure-sed pproch. In Proc. 24 ACM SIGMOD Int. Conf. on Mngement of Dt (SIGMOD 4), Pris, Frnce, June 24. [11] Y. Yn nd J. Hn. gspn: Grph-sed sustructure pttern mining. In Proc. 22 Int. Conf. on Dt Mining (ICDM 2), Meshi, Jpn, Decemer 22. [12] M.J. Zki. Efficiently mining frequent trees in forest. In Proc. 22 Int. Conf. on Knowledge Discovery nd Dt Mining (KDD 2), Edmonton, Alert, Cnd, July 22.