Combinatorial Testing for Tree-Structured Test Models with Constraints

Transcription

1 Comintoil Testing fo Tee-Stutued Test Models with Constints Tkshi Kitmu, Akihis Ymd, Goo Htym, Cyille Atho, Eun-Hye Choi, Ngo Thi Bih Do, Yutk Oiw, Shiny Skugi Ntionl Institute of Advned Industil Siene nd Tehnology (AIST), Jpn, Emil: {tkitmu, tho, ehoi, Univesity of Innsuk, Austi, Emil: Omon Soil Solutions Co, Ltd, Jpn, Emil: {goo htym, shiny Posts & Teleommunitions Institute of Tehnology, Vietnm, Emil: Astt In this ppe, we develop omintoil testing tehnique fo tee-stutued test models Fist, we genelize ou pevious test models fo omintoil testing sed on nd-xo tees with onstints limited to syntti suset of popositionl logi, to llow fo onstints in full popositionl logi We pove tht the genelized test models e stitly moe expessive thn the limited ones Then we develop n lgoithm fo omintoil testing fo the genelized models, nd show its oetness nd omputtionl omplexity We pply tool sed on ou lgoithm to n tul tiket gte system tht is used y sevel lge tnspottion ompnies in Jpn Expeimentl esults show tht ou tehnique outpefoms existing tehniques I Intodution Comintoil testing (CT) is expeted to edue the ost nd impove the qulity of softwe testing [16] Given test model onsisting of list of pmete-vlues nd onstints on them, CT tehnique lled t-wy testing equies tht ll omintions of vlues of t pmetes e tested t lest one Test genetion fo t-wy testing is n tive eseh sujet Consequently, vious lgoithms nd tools with diffeent stengths hve een poposed so f, e g, etg [7], ts [23], [22], s [10], pit [8], it-h [19], nd lot [21] The Clssifition Tee Method (tm) [12], [17], [5], [14], [15] is stutued tehnique fo test modeling in CT The method uses lssifition tees nd popositionl logi onstints to desie test models The effetiveness of CT in ptie hevily depends on the qulity of test modeling, while it is diffiult tsk equiing etivity nd expeiene of testes [11], [1] tm is expeted to e n effetive tehnique to the impotnt tsk, nd is key tehnique in CT [15] Algoithms tht genete t-wy tests fo suh tee-stutued models deseve futhe investigtion Tool te-xl [14] genetes 2- o 3-wy tests fo tm In tht wok [14], the mehnism of test genetion is explined fo simple tee without onstints; howeve, ou inteest is in mehnisms fo genel tees with onstints Inspied y Oste et l [20], we [9] took tnsfomtion ppoh This ppoh tnsfoms tee-stutued test models to non-stutued ones nd feeds them to stndd CT tools suh s the foementioned This wok ws done when the 2nd nd 6th uthos wee in AIST ones [7], [8], [10], [19], [22], [21] This ppoh hs le dvntge: We n levege eent nd futue dvnes in stndd CT tools Howeve, the tehnique [9] inheits the limittion fom pevious tnsfomtion ppohes [20]; they onfine the onstints to syntti suset of popositionl logi The gol of this ppe is to povide tnsfomtion tehnique fo tee-stutued test models with onstints in full popositionl logi, whih we ll T pop To this im, we povide the following ontiutions: 1) We fist exmine dietion of tnslting test models of T pop to one whih [9] n hndle (lled T m ) Unfotuntely, we onlude this dietion is not fesile; we pove tht T pop is stitly moe expessive thn T m 2) Motivted y this ft, we develop tnsfomtion lgoithm dedited to T pop We pove the oetness of ou lgoithm, showing tht the semntis of test models e peseved, nd the t-wy ovege is ensued 3) We futhe nlyze the untime omplexity of ou lgoithm We show tht ou lgoithm hieves omplexity O( N φ ), signifintly impoving O( N 4 ) stted y elie wok [9], whee N is the nume of nodes nd φ is the length of the onstints in test model 4) We implement the lgoithm nd ondut expeiments in n industil setting showing ou tehnique outpefoms the test genetion tool fo tm [14] This ppe is ognized s follows Setion II gives n oveview of the poposed tehnique Setion III defines T pop Setion IV investigtes the expessiveness of T m nd T pop In Setion V, we explin the tnsfomtion lgoithm dedited to T pop Setion VI shows expeimentl esults of ou tehnique in ompison with te-xl Setion VII disusses elted wok, nd Setion VIII onludes II Tnsfomtion Appoh In this setion, we oveview the tnsfomtion ppoh fo tee-stutued test models Fig 1 shows lot test model of hging IC ds in tiket gte system fo ilwy sttions The test model onsists of two pts: n ndxo tee desiing pmete-vlues, nd popositionl-logi onstints on them

2 Chge IC d d ttiute hge with d vendo edit d ge pyment method hge mount A B C edit d ompny with without pyment shedule senio dult hild y edit d (CC) y sh 1KJPY vlid 2KJPY invlid 5KJPY VISA JCB euing one-time CONSTAINTS: (with hild) (y_cc with) (with (A B)) Fig 1 A test model fo n IC Cd hge funtion 1 # List of pmetes nd vlues 2 d vendo: A, B, C 3 edit d: with, without 4 ge: senio, dult, hild 5 pyment method: y CC, y sh 6 hge mount: vlid, invlid 7 pyment shedule: euing, one -time, - 8 edit d ompny: VISA, JCB, - 9 vlid: 1KJPY, 2KJPY, 5KJPY, # CONSTRAINTS 12 IF [pyment method] = "edit d" 13 THEN [edit d] = "with"; 14 (NOT ([ge] = "hild" AND [edit d] = "with")); 15 IF [edit d] = "with" 16 THEN ([d vendo] = "A" OR [d vendo] = "B"); 17 IF [edit d ompny] = "-" 18 THEN (NOT [edit d] = "with") 19 ELSE [edit d] = "with"; 20 Fig 3 PICT ode fo Fig 2 The nd-xo tee desies the si stutue of the test model y deompositionl nlysis of the input domin In Fig 1, the input domin is fist deomposed into two othogonl test spets: d ttiute nd hge with Futhe nlysis deomposes the fome into thee spets: the d vendo, the vilility of edit d funtion, nd the ge of the d holde An ed edge denotes n xoomposition, while its sene epesents n nd-omposition Suh tees speify test models y egding eh xo-node s pmete (lssifition in tm) nd its hilden s the vlues (lsses) of the pmete The popositionl logi onstints desie dependeny mong pmete-vlues in test model The onstints in Fig 1 expess the following: 1) (with hild): A hild nnot hve d with edit d funtionlity 2) y CC with: To py y edit, the d must hve edit d funtionlity 3) with (A B): edit d funtionlity is ville only when the IC d vendo is A o B, ut not C Ou tnsfomtion ppoh, using flttening lgoithm, tnsfoms the tee-stutued test model of Fig 1 to the flt test model of Fig 2 Note tht sevel ext nodes e intodued nd onstints e mnipulted, in ode to keep the semnti equivlene The flttened tee is onveted to stndd fomt of CT, e g, the pit ode in Fig 3 Coespondene etween Fig 2 nd the pit ode is stightfowd: Eh xo-node t the seond level of the tee is pmete in the pit model, nd hilden of n xo-node e vlues of the pmete Tle I shows 2-wy test suite fo this model geneted y pit III Syntx nd Semntis of T pop The test modeling lnguge T pop, whih is genel fom of tee-stutued test models inluding lssifition tees, desies nd-xo tees nd onstints using full popositionl logi The syntx of T pop is defined s follows: Definition 1 (T pop ) The lnguge T pop is the set of tuples φ) s t (N,, ) is ooted tee, whee N is the set of nodes, is the oot node, nd ssigns eh node the set of its hilden ssigns eh node its node {nd, xo, = lef if nd only if = ; φ desies onstints s popositionl fomul given y the following BNF: φ ::= tue flse ( N) φ φ φ φ φ φ φ We ll N n nd-node, xo-node, o lef-node if it is ssoited with nd, xo, o lef, espetively We denote the pent of y, if is not the oot node We ll n element of T pop test tee/model (of T pop ) The semntis of test tee in T pop is defined s set of test ses By defining notion of onfigution of T pop, we n deive test ses fom it Definition 2 (Configution) A (vlid) onfigution of test tee s = φ) is suset C N of nodes tht stisfies the following onditions: 1) The oot node is in C: C 2) If non-oot node is in C, then so is its pent: C C 3) If n nd-node is in C, then so e ll of its hilden: ( = nd ) ( C ) 4) If n xo-node is in C, extly one of its hilden is in C: ( = xo ) (! C ) 5) C stisfies the onstint φ, i e, C = φ We denote the set of ll onfigutions of s y C(s) 2

3 Chge IC d d vendo edit d ge pyment method hge mount vlid edit d ompny pyment shedule A B with senio hild without dult edit sh vlid 1KJPY 1 2KJPY VISA invlid 5KJPY one-time 3 2 JCB euing (with hild) (y_cc with) (with (A B)) ( 1 vlid) ( 2 with) ( 3 with) Fig 2 The flttened tee fo hge IC d TABLE I A pi-wise test suite otined fom the test model in Fig 1 No vendo edit d ge pyment method hge mount edit d ompny pyment shedule vlid 1 B with senio y CC invlid JCB one-time - 2 A with dult y sh vlid VISA euing 1KJPY 3 C without hild y sh vlid - - 3KJPY 4 B with senio y sh vlid VISA one-time 3KJPY The definition of test ses ssumes the si setting of CT Let P e set of pmetes (lssifitions) whee eh p P is ssoited with set V p of vlues (lsses) of p, then test se is vlue ssignment γ to P, i e, γ(p) V p fo evey p P Fo test tee s T pop, we intepet eh xo-node p of s s pmete (p P) nd its hilden s the vlues of the pmete Some pmetes my e sent in test se, sine some xo-nodes my so in onfigution To expess this, we dd speil vlue to V p fo suh pmete p, epesenting in Tle I Definition 3 (Test ses fo T pop ) Let s = φ) e test tee, nd C onfigution of s The test se t C of C is the mpping on P = {p = xo} defined s follows: { v s t v p C if p C t C (p) = if p C Note tht t C (p) is uniquely defined due to ondition 4 of Definition 2 The set of ll the test ses of s is lled the test suite of s nd denoted y s ; i e, s = {t C C C(s)} Exmple 1 Thee e 165 onfigutions tht stisfy the test tee in Fig 1 oding to Definition 2 The highlighted nodes in Fig 1 onstitute suh onfigution, sy C By Definition 3, this onfigution indues the following test se t C, whih oesponds to the fist test se in Tle I: d vendo B, edit d with ge senio, pyment method y CC, t C = hge mount invlid edit d ompny JCB, pyment shedule one-time vlid Definition 4 (t-tuples nd t-wy test suite) Let s e test model nd t positive intege A t-tuple (of vlues) is vlue ssignment on t pmetes, i e, mpping τ : π N suh tht π is set of t xo-nodes nd τ(p) p { } A t-tuple is possile if it ppes in s nd foidden othewise A t-wy test suite of s is set of test ses tht oves ll possile t-tuples of s t lest one IV Expessiveness of T m nd T pop The im of this ppe is to develop tnsfomtion tehnique fo test genetion fo T pop models Thee e two plusile options fo it Option () is to onvet T pop model to n equivlent T m model nd then pply the pevious tehnique in [9] (i e, using the flttening lgoithm fo T m ) Option () is to develop new flttening lgoithm dedited to T pop This setion shows tht ppohes with option () is infesile o disdvntgeous A Coespondene nd Expessiveness Fist, we povide the notion of expessiveness of test modeling lnguges We define it efeing to simil notion povided in [13] 1 ; howeve hee, we onside it sed on oespondene of test suites up to enming Definition 5 (Coespondene) Let Γ nd Γ e sets of test ses on pmetes P nd P, espetively We sy tht Γ oesponds to Γ, denoted y Γ Γ, if nd only if thee exist ijetions p : P P nd vl p : V p V p(p) fo ll p P tht indue ijetion mp : Γ Γ whih is defined s follows: mp(γ) = γ s t γ (p(p)) = vl p (γ(p)) Exmple 2 Conside the test models s nd s T pop in Fig 4 By Definition 3, s = {γ 1, γ 2, γ 3 } nd s = {γ 1, γ 2, γ 3 }, whee γ 1 = { foo, 1} γ 1 = {x 2, y α} γ 2 = { foo, 2} γ 2 = {x 2, y β} γ 3 = {, 3} γ 3 = {x 1, y } 1 In [13], it is defined sed on oespondene of onfigutions without enming 3

4 Fig 4 A smll exmple fo s s Fig 5 A test model tht nnot e expessed in T m Then s s Tht is, we n find ijetions p nd vl tht indue ijetion mp defined in Definition 5, s follows: { x (if p = ) p(p) = y (if p = ) { α (if v = 1) 2 (if v = foo) vl (v) = vl 1 (if v = ) (v) = β (if v = 2) (if v = 3) One my think tht oespondene should e heked egding the t-wy tests fo ll t ( the nume of pmetes) Howeve, this is not needed, s the next theoem sttes: Theoem 1 Suppose tht s s is deived y ijetion mp, nd Γ t-wy test suite of s Γ = {mp(γ) γ Γ} is t-wy test suite of s Poof We show tht ny possile t-tuple τ of s is oveed y Γ Tke ny possile t-tuple τ of s Let τ e the t-tuple of s whih oesponds to τ, i e, mp(τ) = τ (hee, mp is ntully extended fo t-tuples) Fist, we show tht τ is possile in s Sine τ is possile, it ppes in some test se δ s By ssumption, we hve oesponding test se δ s s t mp(δ) = δ Beuse δ s, ll tuples in δ e possile inluding the oesponding t-tuple τ of s Next, we show tht τ is oveed y Γ Sine Γ oves ll t-tuples, thee must exist test se γ Γ tht oves τ It is ovious tht mp(γ) oves τ, nd hene τ is oveed y Γ Definition 6 (Expessiveness) Let T nd T e lnguges We sy T is t lest s expessive s T, denoted y T T, if fo ny s T thee exists s T s t s s We sy T is (stitly) moe expessive thn T, denoted y T > T, if T T nd T T ; nd T is s expessive s T, denoted y T T, if T T nd T T Poposition 1 The eltions, nd < e tnsitive B T pop is moe expessive thn T m, i e, T m < T pop Hee, we ompe the expessiveness of T m nd T pop We fist evisit T m [9] Definition 7 (T m ) The lnguge T m is the suset of T pop onsisting of tuples of fom s = φ m ), whee φ m is onjuntion of fomuls of fom ( ) o with, N We ll onstint of the fome o ltte fom - o eq -onstint, nd wite o eq, espetively Hee we povide thee lemms, whose poofs e found in Appendix The fist lemm tells tht n nd-node oinides with ll its hilden in onfigution Lemm 1 Let s = φ) e n nd-xo tee nd C e onfigution of s Fo ny n N nd n n s = nd, n C if nd only if n C The next lemm sttes tht nodes tht do not ou in ny onfigution n e emoved fom test tee Lemm 2 Let t 1 = φ m ) T m, nd N N e the set of nodes whih neve ppe in C(t 1 ) Let t 2 T m e the tee otined fom t 1 y emoving ll nodes in N nd ll eq nd -onstints whih involve nodes in N, i e, nd eq fo some N o N Then C(t 1 ) = C(t 2 ) The following lemm ensues tht ny sutee tht does not ontin xo-nodes n e edued to single node y mnipultion of onstints Lemm 3 Let t 1 T m, nd t 1 e sutee of t 1 tht does not ontin xo-nodes We denote the oot node of t 1 y n nd the set of nodes of t 1 y N Then t 1 = t 2, whee t 2 is otined y sequentilly pplying to t 1 the following: 1) Fo eh eq -onstint eq in t 1, eple it with n eq if N \ {n} nd y eq n if N \ {n} 2) Fo eh -onstint in t 1, eple it with n if N \ {n} nd y n if N \ {n} 3) Remove ll the nodes in N \ {n} fom t 1 The following theoem shows tht the tnsfomtion tehnique developed fo T m [9] is not pplile to T pop in genel The poof exemplifies test model in T pop nd shows tht it nnot e expessed in T m Theoem 2 T pop is moe expessive thn T m, i e, T m < T pop Poof It is esy to show tht T m T pop, sine nd eq e meely ( ) nd espetively In the following, we show T pop T m y giving tee s T pop whih nnot e expessed in T m Tke the tee in Fig 5 s s Thee e the following seven test ses in s : γ 1 = { 1, 1, 1 } γ 2 = { 1, 1, 2 } γ 3 = { 1, 2, 1 } γ 4 = { 1, 2, 2 } γ 5 = { 2, 1, 1 } γ 6 = { 2, 1, 2 } γ 7 = { 2, 2, 1 } Assume tht thee exists tee s T m suh tht s s is deived y ijetions p nd vl Let us wite p(p) = p, vl(v) = v, nd mp(γ) = γ Note tht 1 nnot e n nesto of, sine in tht se Definitions 2 nd 3 impose γ ( ) 1 γ ( ) = fo evey γ s, whih ontdits eithe the test se γ 3 o γ 7 Similly, 2 nnot e 4

5 1 2 x 1 2 y 1 2 z Fig 6 Requied tee stutue fo test models tht oespond to Fig 5 n nesto of Thus, we know LCA(, ), whee LCA(, ) expesses the lest ommon nesto of nd Anlogously, LCA(, ) nd LCA(, ) Moeove, the xo-nodes in s e extly,, nd, sine thee n e only these thee pmetes in test ses of s This lso entils tht i, i, nd i e not Sine γ 1, γ 5 s with γ 1 ( ) = 1 nd γ 5 ( ) = 2, it follows tht 1, 2 in s Anlogously, 1, 2 nd 1, 2 We onlude tht the stutue of s is s shown in Fig 6, whee lk sutees ontin only nd- o lef-nodes Dshed nodes x, y, nd z must not ppe in ny test se, s s ontins only 1, 2, 1, 2, 1, nd 2 Thus, we onside the tee in Fig 5 insted of Fig 6, s ensued y Lemm 2 nd Lemm 3 If s hs no onstint, then s ontins eight test ses On the othe hnd, s hs seven test ses; thus s s If s inludes t lest one onstint, then the size of s is eithe 8 o t most 6 Why? Let α e the nume of emoved test ses y dding one onstint to s In the se fo u eq v, α = 0 if v Y u = v 2 if v X u X u v 4 if v X u Y 8 if v X u X u = v u v whee X = { 1, 2, 1, 2, 1, 2 } nd Y = {,,, } Fo -onstint, α is eithe 2, 4, o 8 Note tht y dding nd/o eq -onstints, the nume of test ses nnot inese Hene the size of s nnot e 7, nd s s Theoem 2 shows the impossiility of tnsltion of T pop model to n equivlent T m model sed on the defined notion of equivlene in Definition 6, nd hene the infesiility of option () Hee, one my think of nothe eloted tehnique fo suh tnsltion, tht my enle the tnsfomtion ppoh of option () The tehnique is to intodue ext dummy nodes to elize tnsltion of T pop to T m peseving equivlene Tht is, dditionl nodes e intodued in test models of T m to elize suh tnsltion of T pop to T m models, nd these nodes e lso tken into ount lso fo the genetion of t-wy omintoil test suites, ut finlly e filteed out in the esulting test suite Howeve, in this ppe we do not tkle this ppoh, t lest s the fist ttempt to hieve ou gol It is sine we guess this ppoh with suh n elotion fo stiking to option () is less dvntgeous thn option () The fist eson fo this is tht dditionl dummy nodes intodued Algoithm 1: φ; ) Input: A tee s = φ) in T pop nd tget N Output: A tee s in T pop, without nd-sequenes elow 1 foeh do 2 φ) φ; ) 3 = nd then 4 φ φ 5 \ {} ; N N \ {} 6 etun φ) fo tnsltion in option () eome dummy pmetes nd vlues in stndd test models poessed fo genetion of t-wy omintoil test suites These ext elements my use undesile side effets of genetion of lge sized test suites thn neessy s well s highe omputtion osts fo test genetion Contily, the tehnique of option (), tht develops flttening lgoithm fo T pop, n do without suh n elotion, nd thus n void suh side effets The seond eson is tht, s we will show in Setion V-D, the flttening lgoithm fo T pop developed in option () hieves omplexity O( N φ ), whih is lowe thn tht of the lgoithm fo T m [9] nlyzed s O( N 2 φ ) V Flttening Algoithm fo T pop Motivted y the esult in the pevious setion, we develop flttening lgoithm fo T pop tht inputs test tee in T pop nd then tnsfoms it into n equivlent flt one in Tpop f We lso show its oetness poof nd omplexity nlysis A Outline of the flttening lgoithm Fist we define the flt test tees s follows: Definition 8 (Tpop) f Lnguge Tpop f is sulss of T pop s t (1) the oot node is n nd-node, (2) ll the nodes in the seond level e xo-nodes, nd (3) ll the nodes in the thid level e lef-nodes The height of tee in T f pop is lwys two; hene we ll suh tees flt Flt test tees hve the sme stutue s test models in existing omintoil testing tools, suh s pit [8], ts [23], [22], it-h [19], etg [7], nd lot [21] Thus, tests n e geneted y feeding flttened test model to these tools Next, we show tht evey test tee in T pop n e tnsfomed to flt one In othe wods, we pove the following: Theoem 3 T pop T f pop The poof of this theoem is to demonstte the oetness of the tnsfomtion, hieved y lgoithm fltten we develop The lgoithm pplies thee su-lgoithms, emove-nd-seq, emove-xo-seq, nd lift in this ode, ut the ode of the fist two n e swpped Note tht we only onside test tees whose oot node is n nd-node hee; test tees whose oot node is n xo-node n e hndled simply y inseting n nd-node ove the oot [9] 5

6 d e d e φ d Fig 7 emove-nd-seq φ d e e φ φ Algoithm 3: lift(s) 1 funtion lift(s) Input: A tee s = φ) in T pop without nd- nd xo- sequenes Output: A tee s in T f pop 2 O ; H ; ψ tue // glol viles 3 foeh do φ; ) 4 O 5 etun (N φ ψ) 6 sufuntion φ; ) 7 foeh do 8 foeh do 9 φ) φ; ) 10 H H { } whee is fesh node lef; { }; O O {} 12 ψ ψ ( ); φ φ 13 lef Fig 8 emove-xo-seq Algoithm 2: φ; ) Input: A tee s = φ) in T pop nd N Output: A tee s in T pop, without xo-sequenes elow 1 foeh do 2 φ) φ; ) 3 = xo then 4 N N { } whee is fesh node nd; {}; \ {} { } 6 etun φ) d f f d e e φ φ ( ) Fig 9 The min poess of lift(s) Algoithm emove-nd-seq (see Algoithm 1 nd Fig 7), emoves ll nd-sequenes elow given tget node in tee s When onseutive nd-nodes nd e found, it deletes the seond nd-node Then, ll the ouenes of the seond nd-node in φ e epled y its pent By eusively visiting ll the nodes of the input tee stting fom the oot, ) emoves ll ndsequenes fom the input tee Algoithm emove-xo-seq (see Algoithm 2 nd Fig 8) emoves xo-sequenes in given tee, ut it does so diffeently fom emove-nd-seq When it finds onseutive xo-nodes nd, fesh nd-node is inseted in etween This ppoh diffes fom othes [20], [9], whih emove the seond xo-node in n xo-sequene As the oespondene etween two test suites equies ijetion etween thei pmetes, deleting xo-nodes (i e, pmetes) is not ppopite in this setting It is not ovious if t-wy test suite is peseved, even when the nume of pmetes hs hnged On the othe hnd, ou lgoithm is shown to peseve this equivlene (see Theoem 1 nd Theoem 4) Algoithm lift is pplied fte the pevious two lgoithms; it equies tht the input tee e fee of nd- nd xo-sequenes (i e, tees whee nd-nodes nd xo-nodes ppe ltentely) If thee exists n xo-nd-xo-sequene of nodes,, nd, then the seond xo-node is lifted to diet hild of the oot node Howeve, this opetion uses to lwys ppe in onfigution, whih should not e the se if its pent is not in the onfigution Thus, evey ouene of in onstint φ is epled y, whih is equivlent oding to Lemm 1 Futhemoe, in ode to expess the se whee is not in the onfigution, n ext lef node is dded s hild of (see Fig 9) The new option should e hosen if nd only if is not in the onfigution, whih is ensued y dding onstint s onjuntion to onstint φ Algoithm 3 uses the su-lgoithm lift-su, whih tkes tee s nd n xo-node, inditing the tget of lifting It eusively pplies itself to gndhilden of, whih e gin xo-nodes due to the pe-ondition We pepe thee glol viles O, H, nd φ, in ode to stoe lifted xonodes (e g in Fig 8), newly-eted lef-nodes (e g ), nd fomuls dded in the poedue (e g ), espetively B Coetness poof The oetness of ou new lgoithm is stted fomlly s follows: Theoem 4 Fo s T pop, s fltten(s) The theoem is poved y showing the oetness of the thee su-lgoithms emove-nd-seq, emove-xo-seq, nd lift Hee we pesent oetness poof only fo the lift lgoithm Coetness poofs of the othe two, whih e not s ovious s one my expet, n e found in Appendix 6

7 Lemm 4 Fo s T pop without nd- nd xo-sequenes, s lift(s) Poof The poess of lgoithm lift is n itetion of lifting eh xo-node t the thid level to the fist level (see lines 7 15 in Algoithm 3) We pove tht evey time the poedue is pplied, the set of test ses emins unhnged Suppose tht the poedue is pplied to n xo-nd-xo sequene of nodes,, nd in tee t 1 T pop (left pt of Fig 9), yielding tee t 2 (ight pt of Fig 9) Let us split the set of onfigutions fo t 1 into the following two susets: The set of onfigutions (1) without : {C C(t 1 ) C} nd (2) with : {C C(t 1 ) C} We split t 2 nlogously: The set of onfigutions (3) without : {C C(t 2 ) C} nd (4) with : {C C(t 2 ) C} Bsed on this se nlysis, the following n e indued: (i) (2) nd (4) e equivlent In oth sets (2) nd (4), is inluded in the onfigutions This mens tht is inluded in oth sets of onfigutions s well The only diffeene tht my ise in suh sitution is tht my e inluded in the onfigutions in (4), ut not in the onfigutions in (2) Howeve, this diffeene neve ous, due to the dded onstint (ii) The set of onfigutions otined y dding nd to eh onfigution in (1) is equivlent to (3) Sine is not inluded in ny of these onfigutions, we n sfely emove the su-tees elow in oth t 1 nd t 2 Then, the only diffeene tht my ise etween suh tees is tht hild d i of my ppe in onfigution C in (3) ut not in (1) Howeve, due to the dded onstint, C Hene, d i nnot ppe in C due to Definition 2 On the othe hnd, nd e lwys inluded in the onfigutions Test ses e peseved in oth (i) nd (ii) Fo (i), the sets of onfigutions in t 1 nd t 2 e equivlent, nd the diffeenes in the tee stutues etween t 1 nd t 2 do not ffet test ses (sine they do not ffet the eltionship etween xo-nodes nd thei hilden) Fo (ii), the only diffeene etween the sets of onfigutions (2) nd (4) is tht is inluded in (4) ut not in (2) Sine is not in (2), t C () = fo evey C in (2) Hene, we onlude the lim y onsideing in Definition 3 the identity p nd the following vl p : { if p = nd x = vl p (x) = x othewise C Complexity nlysis In this setion, we nlyze the untime omplexity of ou new flttening lgoithm We stte the min esult fist Theoem 5 The omputtionl omplexity of fltten(s) is O( N φ ) fo s = φ) The poof is done y showing the omplexity of the thee su-lgoithms Hee we only show the omplexity nlysis fo lift, whih is the most inteesting pt The othe two n e nlyzed in stightfowd mnne Lemm 5 The omplexity of φ) is O( N φ ) Poof Let T(s) denote the omplexity of lift(s) nd T (s, ) tht of lift-su(s; ) Fist, we show tht T (s, ) is O( N φ ) y indution on N, whee N denotes the set of the nodes elow The lgoithm lift-su itetively pplies the opetions of lines to ll gndhilden 1,, k of the given node The wost-se omplexity of line 14 is O( φ ), while lines n e done in onstnt time Aodingly, line 14 domintes the omplexity of this poedue We otin the following, whee eh N i is the set of the nodes elow i : T (s, ) = T (s, 1 ) + O( φ ) + + T (s, k ) + O( φ ) = T (s, 1 ) + + T (s, k ) + t O( φ ) By pplying the indution hypothesis to eh T(s, i ), we poeed s follows: = O( N 1 φ ) + + O( N k φ ) + t O( φ ) = (O( N 1 ) + + O( N k )) O( φ ) + t O( φ ) = O( N N k + t) O( φ ) Finlly, lift lls lift-su fo ll hilden 1,, k of the oot node Thus, the omplexity of lift is s follows, whee N i denotes the set of nodes elow i : T(s) = T (s, 1 ) + + T (s, k ) = O( N 1 ) O( φ ) + + O( N k ) O( φ ) = O( N N k ) O( φ ) Sine N 1 N k = N \ {, 1,, k }, we onlude T(s) = O( N φ ) The omplexities of the othe two su-lgoithms e poved in simil mnne In summy, we hve the omplexity of emove-nd-seq: O( N φ ), the omplexity of emove-xo-seq: O( N ), nd the omplexity of lift: O( N φ ) Hee we ssume set opetions tht dd o emove one element e omputle in onstnt time Note tht emove-nd-seq nd emove-xo-seq keep the nume of nodes within O( N ) nd the length of onstints within O( φ ) fo input tee s = φ) This is equied to guntee the omplexity of the entie poedue D Complexity nlysis in ompison The omputtionl omplexity of the new lgoithm, onluded s O( N φ ), is signifintly lowe thn tht of ou pevious wok [9] s stted O( N 4 ) Thee e two esons fo this One eson is tht ou new lgoithm hs le eusive stutue tht llows n ute omplexity nlysis Afte simil efinement, the omplexity of the lgoithm in [9] would e O( N 2 φ ) The othe eson is n impovement of the lgoithm emove-xo-seq Fig 10 illusttes the si poess of the emove-xo-seq ountept in [9] (using nottions of the uent ppe) We n oseve tht it ollpses xo-sequenes, 7

8 1 1 2 n 2 n REQ φ (x ) em-o-seq REQ REQ φ (x 1 ) (x n ) Fig 10 emove-xo-seq(s) in [9] ut this omes t the ost of mking the size of onstints lge y the ode of the size of input test tees Moe speifilly, this esults in n output test tee with onstints φ whose size is O( N φ ) 2 Tees output y emove-xoseq is pssed to the downwd su-lgoithm φ ), whose omplexity is O( N φ ) Hene, we hve O( N φ ) = O( N ( N φ )) = O( N 2 φ ) On the othe hnd, the uent vesion of emove-xo-seq of the flttening lgoithm fo T pop keeps the length of onstints within O( φ ) s well s the nume of nodes within O( N ) fo input tee s = φ) VI Expeimentl Results We hve implemented the flttening lgoithm (in C++) in ou tool lot, whih is then omined with pit (ve 33) nd ts (ve 29) Expeiments ompe lot with Clssifition Tee Edito (te-xl, ve 35) [12], [17], [14], test geneto fo tm nd the only ompetito tht n poess tee-stutued test models s f s we know (see Setion VII) As enhmk set, we desied test models in lot nd in tm fo 18 API funtions of n tul tiket gte system Expeiments wee pefomed on mhine with Intel Coe i7-4650u CPU 170G Hz with 8 GB Memoy nd Windows 7 Tle II shows the esults of ou expeiments fo 2-wy nd 3-wy testing Columns 2 nd 3 show the sizes of the input models in T pop, in tems of the nume of nodes nd the size of onstints ( N nd φ ) Columns 4 nd 5 show the sizes of the flttened models, mesued y the size of the model nd the onstints ( M nd φ ) Fo exmple, test model 1 witten in T pop ontins 120 nodes nd the size of the onstints is 50 It is flttened to model M = with onstints of size φ = 90; hee x y epesents tht the model hs y pmetes with x vlues The ight pt ompes lot with pit, lot with ts, nd te-xl, in tems of nume of geneted test ses nd exeution times (in seonds) Fo the numes of test ses, the smllest ones e highlighted fo eh test model Exeution time fo lot inludes the time fo flttening nd tht fo test genetion y eithe pit o ts, lthough the time fo flttening ws less then 003 seonds fo ll models Fo 2 Hee, note tht we nnot hve the following deivtion, sine the deived logil fomul is not legitimte onstint of T m : φ ((x eq 1 ) (x eq 2 ) (x eq n )) φ (x eq ( 1 2 n )) n te-xl, sine it only hs GUI, we mesued the exeution time with stopwth Timeout is set to 3,600 seonds The esults show tht lot outpefoms te-xl w t the nume of geneted test ses, with one inteesting exeption of No 3 fo 2-wy testing In some test models, the diffeene is onsidele; e g in test model 4 fo 3-wy testing, the nume of test ses lot genetes is six times smlle thn wht te-xl yields Regding exeution time, lthough the esults fo te-xl e inute, lot exels ove te-xl y n ode of mgnitude in quite few exmples Agin thee is one exeption (No 2 fo 2-wy testing), ut in this se te-xl genetes thee times s mny test ses s lot does In ddition, it is woth noting tht te-xl does not suppot t-wy testing with t 4, while lot suppots ny t tht the k-end engine dmits Moeove, little implementtion effot enles futhe extensions in lot fo othe ovege itei, e g, see [8] VII Relted Wok Test genetion fo t-wy testing is n tive eseh sujet in CT Consequently, vious lgoithms nd tools with diffeent stengths hve een poposed so f, e g, etg [7], ts [18], [23], s [10], pit [8], sde [24], ith [19], nd lot [21] Fo exmple, the lgoithm in [22] n effiiently hndle test models with omplex onstints The lgoithm in [21] speilizes in minimizing the t-wy test suite within n llowed time Ou tnsfomtion ppoh n enjoy suh vious stengths of diffeent lgoithms Note lso tht ll of these lgoithms nd tools, exept fo te-xl, nnot dietly hndle tee-stutued test models te-xl [14] is the only tehnique, exept fo lot, tht n poess tee-stutued test models fo t-wy testing Thei test models nd ou T pop e essentilly equivlent; te-xl uses solled lssifition tees, while lot uses nd-xo tees The mjo diffeene ppes in the lgoithms fo test genetion Although the lgoithm in te-xl hs not een eveled in detil, fom ptil explntion in [14], we n oseve tht it dietly onstuts test ses fom tee-sed test model Thus, unlike lot, te-xl nnot enefit fom the vious lgoithms nd tools fo stndd t-wy test se genetion Supeioity of the lot ppoh is demonstted though expeiments in Setion VI We hve peviously developed estited tnsfomtion lgoithm fo tee-stutued test models with onstints [9] The ontiution of this ppe ove tht wok [9] is theefold (1) We develop tnsfomtion lgoithm fo tee-stutued test models with onstints witten in full popositionl logi (lled T pop ), motivted y the ft tht the pemitted onstints in pevious wok [9] (lled T m ) e limited to syntti suset of popositionl logi The poof of T m < T pop in Theoem 2 indites tht pevious lgoithms nnot e used fo ou gol (2) We efine ou omplexity nlysis In [9], we stted tht the omplexity of the pevious lgoithm is O( N 4 ) This ppe shows tht of the new lgoithm to e O( N φ ) (3) Ou se study is nothe impotnt ontiution By elxing 8

9 TABLE II Test se genetion using lot nd te-xl fo the tin tiket gte system 2-wy tests 3-wy tests T pop Flttened model Clot/pit Clot/ts CTE Clot/pit Clot/ts CTE N φ M φ size time size time size time size time size time size time time out time out time out time out time out time out < < < < < < < < < < < < < < < < < < < < < < eo eo limittions on onstints, we e now le to ompe the tnsfomtion ppoh with te-xl Ou wok is inspied y Oste et l [20], who pplied CT to Softwe Podut Lines (SPLs) An SPL is expessed s fetue model, n nd-o tee with onstints Thei flttening lgoithm tnsfoms fetue model to flt one, ( fetue model whose height is two) nd set of poduts is otined s t-wy test suite fo the SPL The lss of fetue models in [20] estits the onstints to the sme syntti suset s T m Also, we n oseve tht simil tehnique to ou flttening lgoithm is used in [3] fo the setting of SPL testing Comped to these wok, ou ontiution evels sevel impotnt fts of the poposed tehnique suh s expessiveness, oetness, nd omplexity nlysis We expet tht ou ontiutions n e lso used to dvne thei ppoh fo SPL VIII Conlusions nd Futue Wok This ppe pesented tnsfomtion ppoh to geneting t-wy tests fo tee-stutued test models with popositionl logi onstints Ptility of ou tehnique ws evluted y expeiments in n industil setting; they demonstted supeioity of ou tehnique ove the stte-of-thet tool te-xl Also, the ppe ontins sevel theoetil ontiutions We poved T m < T pop ; to ou knowledge, thee is no pulished poof fo this non-tivil esult We lso poved the oetness nd omplexity of the lgoithm Pioitized t-wy testing extends t-wy testing with the pioity notion to expess impotne of diffeent test spets [2], [6], [15] Fo tee-stutued test models, te-xl hs ledy intodued pioities [14] Fo futue wok, we pln to onnet the notion of pioity fo stndd t-wy testing nd tht fo tee-stutued testing vi ou tnsfomtion ppoh We will lso investigte the eltion etween tee-stutued test modeling nd omintoil testing with shielding pmetes [4] Shielded pmetes my not ppe in some test ses, depending on whethe some othe pmetes hve speified vlues This fetue my open up new wys to model hiehil dependenies in tee-stutued test models, ut it hs so f not een extensively nlyzed Aknowledgment We e gteful to the nonymous eviewes of the ppe fo thei onstutive nd eful omments The fist utho would like to thnk Hitoshi Ohski fo his suppot nd insightful omments t n ely stge of this wok This wok is in pt suppoted y JST A-STEP gnt AS H Refeenes [1] M N Bozjny, L Yu, Y Lei, R Kke, nd R Kuhn Comintoil testing of ACTS: A se study In Po of ICST 12, pges IEEE, 2012 [2] R C Bye nd C J Coloun Pioitized intetion testing fo piwise ovege with seeding nd onstints Infom Softwe Teh, 48(10): , 2006 [3] A Clvgn, A Ggntini, nd P Vvssoi Comintoil testing fo fetue models using itl In Po of ICST 2013 Wokshops, pges , 2013 [4] B Chen, J Yn, nd J Zhng Comintoil testing with shielding pmetes In Po of APSEC 10, pges , 2010 [5] T Y Chen, P-L Poon, nd T H Tse An integted lssifitiontee methodology fo test se genetion Int J Softw Eng Know, 10(6): , 2000 [6] E Choi, T Kitmu, C Atho, nd Y Oiw Design of pioitized N-wise testing In Po of ICTSS 14, LNCS 8763, pges Spinge, 2014 [7] D M Cohen, S R Dll, M L Fedmn, nd G C Ptton The AETG system: An ppoh to testing sed on omintioil design IEEE Tns Softwe Eng, 23(7): , 1997 [8] J Czewonk Piwise testing in el wold: Ptil extensions to test se senios In Po of PNSQC 06, 2006 [9] N T B Do, T Kitmu, N V Tng, G Htym, S Skugi, nd H Ohski Constuting test ses fo N-wise testing fom tee-sed test models In Po of SoICT 13, pges ACM, 2013 [10] B J Gvin, M B Cohen, nd M B Dwye Evluting impovements to met-heuisti seh fo onstined intetion testing Empiil Softw Eng, 16(1):61 102,

10 [11] M Gindl nd J Offutt Input pmete modeling fo omintion sttegies In in Po of IASTED 07 on Softwe Engineeing, pges ACTA Pess, 2007 [12] M Gohtmnn Test se design using lssifition tees In Po of STAR 1994, 1994 [13] P Heymns, P Shoens, J Tigux, Y Bontemps, R Mtuleviius, nd A Clssen Evluting foml popeties of fetue digm lnguges Po of IET Softwe, 2(3): , 2008 [14] P M Kuse nd M Lunik Automted test se genetion using Clssifition Tees Softwe Qulity Pofessionl, pges 4 12, 2010 [15] D R Kuhn, R N Kke, nd Y Lei Intodution to Comintoil Testing CRC pess, 2013 [16] D R Kuhn, D R Wlle, nd A M Gllo Softwe fult intetions nd implitions fo softwe testing IEEE Tns Softwe Eng, 30(6): , 2004 [17] E Lehmnn nd J Wegene Test se design y mens of the CTE XL In Po of EuoSTAR, 2000 [18] Y Lei nd K-C Ti In-pmete-ode: A test genetion sttegy fo piwise testing In Po of HASE 98, pges IEEE, 1998 [19] T Nn, T Tsuhiy, nd T Kikuno Using stisfiility solving fo piwise testing in the pesene of onstints IEICE Tnstions, 95-A(9): , 2012 [20] S Oste, F Mket, nd P Ritte Automted inementl piwise testing of softwe podut lines In Po of SPLC 10, LNCS 6287, pges Spinge, 2010 [21] A Ymd, T Kitmu, C Atho, E Choi, Y Oiw, nd A Biee Optimiztion of omintoil testing y inementl SAT solving In Po of ICST 15, pges 1 10 IEEE, 2015 [22] L Yu, Y Lei, M N Bozjny, R Kke, nd D R Kuhn An effiient lgoithm fo onstint hndling in omintoil test genetion In Po of ICST 13, pges IEEE, 2013 [23] L Yu, Y Lei, R Kke, nd D R Kuhn ACTS: A omintoil test genetion tool In Po of ICST 13, pges IEEE, 2013 [24] Y Zho, Z Zhng, J Yn, nd J Zhng Csde: A test genetion tool fo omintoil testing In Po of ICST 2013 Wokshops, pges , 2013 A Omitted poofs Appendix Poof of Lemm 1 The poof is stightfowd fom the ondition (3) in Definition 2 Poof of Lemm 2 Let t 1 e the tee otined y dding -onstint n fo eh n N whee is the oot node Let t 1 e the tee otined y emoving ll - nd eq -onstints tht involve ny nodes in N exept fo the onstints dded in t 1 We pove the following: 1) C(t 1 ) = C(t 1 ) Sine n N is node whih neve ppes in ny onfigution of t 1 nd the oot node is lwys in onfigution of t 1, dding suh onstints mkes no diffeene etween C(t 1 ) nd C(t 1 ) 2) C(t 1 ) = C(t 1 ) We onside the following thee foms of the onstint: ) eq with N nd is ny node in t 1 : Sine N, this onstint does not ffet Futhe, sine, still does not ppe in ny onfigution fte emoving eq ) eq with is ny node in t 1 nd N : Sine N, N Hene, the se follows se ) ) with N o N nd oth e not : Anlogous to ) 3) C(t 1 ) = C(t 2) By emoving ll the nodes in N nd ll the -onstints etween the oot node nd nodes in N fom t 1, we otin t 2 The emovl mkes no diffeene etween C(t 1 ) nd C(t 2), sine nodes in N e not efeed to y ny othe onstints in t 1 Poof of Lemm 3 We show tht these poedues do not hnge the test suite Fo 1) nd 2), the lim follows Lemm 1 Fo 3), it is euse (i) the sutee ontins no xo-nodes nd hene ny nodes in N \ {n} do not ppe in the test suite oding to Definition 3, nd (ii) no onstint involves ny node in N \ {n} fte pplying 1) nd 2) Lemm 6 Let s = φ) T pop nd t = φ; ) Then, s = t nd thus s t Poof Algoithm 1 pplies to evey nd-sequene in s the following two steps: 1) sustituting the seond nd-node of the nd-sequene in φ y its pent (line 4), nd 2) deleting the seond nd-node fom the tee (line 5) We show tht eh of these steps peseves the test suite of n input tee 1) This step yields s = φ ) whee φ = φ To pove s s, suppose t C s Due to Lemm 1, we hve C = φ nd thus C C(s ) Hene y definition, t C s Anlogously we hve s s, onluding s = s 2) This step yields s = (N \ φ ) whee { () \ {} () if x = (x) = (x) othewise It is esy to show tht C C(s ) if nd only if C \ {} C(s ), sine does not ou in ondition φ, whih is used y oth s nd s Below we show tht t C = t C whee C = C \ {} Suppose t C (p) = v Then, p sine is not n xo-node, nd v sine is not hild of n xonode Hene, p, v C nd lso v p Aodingly, t C (p) = v Suppose t C (p) = v Then, v p nd p sine is not n xo-node Thus, v p Also, p, v C, sine v C Hene t C (p) = v This onludes s = s Lemm 7 Let s = φ) T pop nd t = φ; ) Then s t Poof Algoithm 2 is n itetion of the poedue shown in Fig 8 Eh time it is pplied to n xo-sequene of N nd in tee s yielding s, we n pove s s y egding the identity p nd the following vl p in Definition 5: { if p = nd v = vl p (v) = v othewise Poof of Theoem 4 Fom Lemms 6, 7, nd 4, eh of the thee steps in the lgoithm fltten peseves test ses up to oespondene 10