Automated Specification-based Testing of Interactive Components with AsmL

Transcription

1 1 utomtd Spcifiction-sd Tsting of Intrctiv Componnts with sml n C R Piv João C P Fri nd Rul F M Vidl strct It is prsntd promising pproch to tst intrctiv componnts supporting th utomtic gnrtion of tst css from spcifiction Th rlvnc nd difficultis issus nd chllngs) ssocitd with th tsting of intrctiv componnts r first prsntd It is shown tht forml spcifiction with crtin chrctristics llows th utomtic gnrtion of tst css whil solving som of th issus prsntd Th pproch is illustrtd with n xmpl of utomtic tsting of th conformity twn th implmnttion of utton in th Nt frmwork nd spcifiction writtn in th sml lngug using th sml Tstr tool Th conclusion discusss th chrctristics of th tool nd givs dirctions for futur work Indx Trms Forml Mthods Intrctiv Systms Tsting 1 INTROUCTION T h dvlopmnt of high-qulity intrctiv systms nd pplictions is difficult nd tim-consuming tsk rquiring xprtis from divrs rs softwr nginring psychology) Currnt IE's r not powrful nough for spcifying/modling uilding nd tsting thos systms in n ffctiv wy Th dvlopmnt of intrctiv systms nd pplictions sd on rusl intrctiv componnts is th ky to chiv highr qulity nd productivity lvls Improving th qulity of intrctiv componnts should hv mjor impct in th qulity of intrctiv systms nd pplictions uilt from thm nd should contriut to thir incrsd rus In this ppr it is prsntd promising pproch to th tsting of intrctiv componnts By intrctiv componnts w mn rusl controls or widgts or intrctors cpl of oth input from th usr nd output to th usr writtn in gnrl-purpos ojct-orintd lngug such s Jv or C# Intrctiv componnts rng from th mor sic ons such s uttons txt oxs como oxs list oxs tc) to th mor sophistictd ons clndrs dt grids intrctiv chrts tc) uilt from simplr ons Th ovrhd incurrd in tsting rusl intrctiv componnts pys-off cus of thir widr usg nd longvity whn comprd to spcil purpos nd short livd "finl" usr intrfcs Th ppr is orgnizd s follows: nxt sction sction 2) prsnts som importnt issus nd chllngs of tsting intrctiv componnts Sction 3 xplins th typ of tst utomtion tht is nvisiond utomtd spcifictionsd tsting) discusss th typ of forml spcifiction rquird nd discusss its costs nd nfits Sction prsnts n xmpl of prforming utomtd spcifictionsd tsts using th sml lngug nd th sml Tstr tool Som conclusions nd futur work cn found in th ll uthors r with th Elctricl Enginring prtmnt Enginring Fculty of Porto Univrsity E-mil: piv@fuppt jpf@fuppt nd rmvidl@fuppt finl sction 2 ISSUES N CHLLENGES OF TESTING INTERCTIVE COMPONENTS Tsting intrctiv componnts is prticulrly difficult cus it shrs nd comins th issus nd chllngs of tsting ojct-orintd systms [1 componnt-sd systms [2 nd intrctiv systms Som of th min issus nd chllngs r idntifid nd dscrid nxt 21 Complx Evnt-drivn Bhviour Intrctiv componnts nd intrctiv pplictions nd systms in gnrl) hv complx vnt-drivn hviour difficult to nlyz nd prdict nd consquntly lso difficult to tst nd dug Evn sic intrctiv componnts such s uttons nd txt oxs my rct to nd gnrt dozns of vnts Most of us hv lrdy xprincd "strng" hviours lockd intrfcs dirty displys tc) pprntly t rndom whn using wid-sprd intrctiv pplictions nd systms This should not surpris givn thir complx vnt-drivn hviour 22 Highly-configurl or Customizl) Bhviour Rusl intrctiv componnts usully hv highlyconfigurl or customizl) hviour This cn don stticlly or dynmiclly y stting configurtion proprtis or ttriuts y dding vnt-hndlrs or y dfining suclsss nd mthod ovrriding Tsting n intrctiv componnt in ll th configurtions llowd is lmost impossil cus of th hug st of possil configurtions nd th difficulty to prdict th customizd hviour 23 Multipl Intrfcs Intrctiv componnts hv oth usr intrfc GUI) nd n ppliction intrfc PI) Th ppliction intrfc is usd for customizing nd composing thm nd for linking thm with th undrlying ppliction logic iffrnt kinds of inputs nd outputs occur vi ths diffrnt intrfcs dqut tsting of n intrctiv componnt cnnot look t just on of ths intrfcs in isoltion nd hs to

2 2 QUTIC 200 PROCEEINGS tk into ccount ll ths kinds of inputs nd outputs in th dfinition of tst css nd in tst xcution 2 GUI Tsting is ifficult to utomt utomting th tsting of grphicl usr intrfcs poss wll-known chllngs: 1 How to proprly simult inputs from th usr mous kyord nd othr highr-lvl vnts tht r gnrtd y th usr) 2 How to chck th outputs to th usr without xcssiv snsitivity to formtting nd rndring dtils 25 PI's with Cllcks nd Rntrnc Th dsignr of rusl intrctiv componnt dfins its mthods ut dos not know in dvnc which kind of pplictions will mk us of thm Mthod clls twn n intrctiv componnt nd n ppliction occur in oth dirctions: 1 Th ppliction or tst drivr) my cll mthods of th intrctiv componnt From th tsting prspctiv inputs r mthods invokd with prmtrs whil outputs r th vlus rturnd y thos mthods This is th trditionl sitution in unit tsting 2 Th intrctiv componnt my gnrt vnts origintd from th usr or intrnlly gnrtd) tht cus th invoction of mthods in th ppliction or tst stu) y som kind of cllck mchnism vnt hndlrs or suclssing nd mthod ovrriding) gin from th tsting prspctiv th outputs r th vnts nd prmtrs pssd to th ppliction whil inputs r rturnd prmtrs Tsting th scond kind of intrction cllcks) poss spcific issus nd chllngs s lrdy notd in [3: 1 n ppliction mthod invokd in cllck my in turn invok mthods of th intrctiv componnt rntrncy sitution) nd hv ccss or chng its intrmdit stt Hnc th intrnl stt of th intrctiv componnt whn it issus cllck is not irrlvnt Morovr som rstrictions my hv to posd on th stt chngs tht n ppliction my rqust whn procssing cllck 2 uring tsting on hs to chck tht: 1) th pproprit cllcks r ing issud 2) whn cllck is issud th intrctiv componnt is put in th pproprit intrnl stt 3) during th procssing of cllck th ppliction dosn't try to chng th stt of th intrctiv componnt in wys tht r not llowd 26 Oprting Systm Intrfrnc Intrction with th usr is mditd y th oprting systm in non trivil wys oftn svrl lyrs r involvd) introducing mor dimnsions of configurility nd complicting th nlysis nd prdiction of its hviour s wll s th tsting nd dugging tsks 27 Insufficint ocumnttion Th documnttion supplid with intrctiv componnts is usully scrc nd not rigorous nough for mor dvncd uss such s dvncd customiztion nd thorough tsting For xmpl from th documnttion it is difficult to know prcisly: 1 whn r vnts signlld nd y wht ordr 2 wht is th intrnl stt of componnt whn it signls n vnt 3 wht is sf for n vnt hndlr to do wht intrctions xist twn vnts This usully lds to tril-nd-rror styl of ppliction progrmming nd poor ppliction qulity nd lso complicts th dsign of tst css 28 Poor Tstility Tsting of intrctiv componnts is usully difficult nd tim-consuming du to: 1 th lck of rigorous unmiguous nd comprhnsiv documnttion 2 th rducd osrvility cpility to osrv th intrnl stt disply producd nd vnts risd) 3 th dficint controllility cpility to simult usr input) Som of th issus nd chllngs dscrid in this sction will ddrssd y our tsting pproch nd discussd in th nxt sctions 3 UTOMTE SPECIFICTION-BSE TESTING Mnul tsting of GUIs nd intrctiv componnts is lour-intnsiv frquntly monotonous tim-consuming nd costly Som of th rsons r th xistnc of vrid possiilitis for usr intrction nd lrg numr of possil configurtions for ch componnt nd othr issus dscrid in sction 2 mking it imprcticl th stisfction of dqut covrg critri y mnul tsting It is ncssry to us som typ of utomtion to prform thos tsts 31 gr of utomtion Envisiond Th dgr of utomtion w nvision is th utomtic gnrtion of tst css inputs nd xpctd outputs) from spcifiction nd not just th typ of utomtion tht is providd y unit tsting frmworks nd tools such s JUnit wwwjunitorg) or NUnit wwwnunitorg) or th typ of utomtion providd y cptur nd rply tools such s WinRunnr wwwmrcurcom) nd othr tools wwwstlscom/mrick/fqs/t-guihtm) Unit tsting frmworks nd tools r of grt hlp in orgnizing nd xcuting tst css prticulrly for PI tsting ut not in gnrting tst css from spcifiction Cptur nd rply tools r proly th most populr tools for GUI tsting ut don t support th utomtic gnrtion of tst css With ths tools it is possil to rcord th usr intrctions with grphicl usr intrfc mous input kyord input tc) nd rply thm ltr Cptur nd rply tools r of grt hlp in svrl scnrios ut lso hv widly rcognizd limittions s g th lsson "cptur rply fils" in [) In this typ of tst utomtion thr is no gurnt of tst covrg nd thr is n xcssiv dpndncy on th physicl dtils of th usr intrfc From highr prspctiv ths diffrnt pprochs nd typs of utomtion r complmntry nd not opponnts

3 3 QUTIC 200 PROCEEINGS Th utomtic gnrtion of tst css from spcifiction rquirs som sort of forml spcifiction of th softwr to tstd tht cn usd to gnrt concrt input vlus nd squncs s wll s th xpctd outputs s tst orcl) It is possil to dsign tst css for lck-ox tsting) from informl spcifictions ut not in n utomtd wy t most inputs cn gnrtd utomticlly sd on th signturs of mthods nd vnts thir clling syntx) ut xpctd outputs cn only gnrtd sd on forml spcifiction of thir smntics 32 Typ of Spcifiction Ndd populr typ of spcifiction of ojct-orintd systms is sd on th principls of dsign y contrct [5 y mns of invrints nd pr nd post-conditions s found in Eiffl wwwifflcom) ContrctJv [6 or JContrct wwwprsoftcom) n invrint is condition tht rstricts th vlid stts of n ojct t lst on th oundris of mthod clls pr-condition of mthod is condition on th input prmtrs nd th intrnl stt of th ojct tht should hold whn mthod is clld On th opposit sid post-condition of mthod is condition on th input prmtrs initil stt of th ojct whn th mthod is clld) finl stt of th ojct whn th mthod rturns) nd vlu rturnd tht should hold t th nd of th mthod xcution lthough with limittions som tst tools such s JTst wwwprsoftcom) hv th cpility of gnrting unit tsts sd on th spcifiction of pr nd post-conditions Whil pr nd post-conditions r good mn to rstrict th llowd hviours of n ojct thy r not dqut in gnrl to fully spcify thir intndd hviour prticulrly whn cllcks r involvd s lrdy notd y Szyprski in his ook [3 th smntics of componnts tht issu cllcks s is th cs of intrctiv componnts s sction 2) cnnot cpturd only y mns of pr nd post-conditions t lst with th mning prsntd ov) Th Ojct Constrint Lngug OCL) s wwwumlorg) gos stp furthr y llowing th spcifiction in th post-condition of mthod of mssgs tht must hv n snt mthod clls nd snding of signls) during its xcution Howvr in gnrl it is not possil to spcify th ordr y which mssgs r snt nd th stt of th ojct whn ch mssg is snt importnt cus of r-ntrnc s xplind in sction 2) Th dfinition of post-conditions in OCL hs nothr dvntg ovr its dfinition in Jv or Eiffl cus OCL is highrlvl forml lngug supporting forml rsoning nd utomtion sml [7 forml spcifiction lngug dvlopd t Microsoft Rsrch tightly intgrtd with th Nt frmwork ridgs ovr th limittions found in OCL y mns of "modl progrms" "modl progrm" is n xcutl spcifiction of mthod modl progrm my orgnizd in squnc of stps For xmpl if mthod issus cllck in th middl of its xcution thr stps should dfind: first stp to ld th ojct to th pproprit stt for issuing th cllck scond stp whr th cllck is issud third stp to ld th ojct to th pproprit finl stt nd rturn Ths stps fcilitt th dfinition of rstrictions on squncs of ctions/vnts tht r common to find in usr intrfc modlling nd r not sy to xprss using just post-conditions Ech stp compriss on or mor non-contrdictory "modl sttmnts" tht r xcutd simultnously Modl sttmnts r writtn in high-lvl ction lngug with primitivs to crt nw ojcts ssign nw vlus to th ttriuts of n ojct nd cll othr mthods Modl progrms my usd in comintion with pr nd post-conditions usully dispnsing th ltr Exmpls of spcifictions writtn in sml will prsntd in sction 33 Conformity Chcks With pproprit tool support s is th cs of th sml Tstr tool) modl progrms cn usd s xcutl spcifiction orcls [1 Tht is th rsults nd stt chngs producd y th xcution of modl progrms xcutl spcifictions writtn in sml) cn comprd with th rsults producd y th xcution of th corrsponding implmnttion undr tst writtn in ny Nt complint lngug in this cs) ny discrpncis found r rportd y th tool Mppings twn ctions nd stts in th spcifiction nd th implmnttion hv to dfind ithr xplicitly or implicitly sd on nm qulity) lthough this is not th only wy of prforming conformity chcks twn spcifiction nd n implmnttion s [8 for discussion of othr possil wys) it is fsil wy 3 Finit Stt Mchin Modl nd Tst Cs Gnrtion For th gnrtion of tst css th sml Tstr tool first gnrts FSM Finl Stt Mchin) from th sml spcifiction nd thn gnrts tst suit with on or mor tst css) from th FSM ccording to critri providd y th usr Sinc th numr of possil ojct stts possil comintions of vlus of instnc vrils) is usully hug th stts in th FSM r n strction of th possil ojct stts ccording to som critri providd y th usr It is wll known tht stt mchin modls r pproprit for dscriing th hviour of intrctiv systms nd rctiv systms in gnrl) nd good sis for th gnrtion of tst css ut usully thr is not good intgrtion twn th ojct modl nd th stt mchin modl sml nd th sml Tstr tool solv this prolm with th gnrtion of th FSM from th spcifiction forml ojct modl) 35 dvntgs of th Forml Spcifiction of Intrctiv Componnts Whn comprd to othr tsting tchniqus utomtd spcifiction-sd tsting hs th disdvntg of rquiring forml spcifiction to chiv highr dgr of utomtion) But th invstmnt in th forml spcifiction of rusl intrctiv componnts my lrgly compnstd y th multipl nfits it cn ring:

4 QUTIC 200 PROCEEINGS 1 Forml spcifictions nd modls r n xcllnt complmnt to informl spcifictions nd documnttion cus miguitis r rmovd nd inconsistncis r voidd 2 Forml spcifictions llow th utomtion of spcifiction-sd tsting s dscrid in this ppr 3 Bsids ing usful s th sis for th gnrtion of tst css FSM's cn lso usd to utomticlly prov rquird proprtis of systm with modlchcking tools tht xhustivly srch th stt spc Th proprtis r writtn in tmporl logic For xmpl Cmpos in [9 uss modl chcking tools to prov usility proprtis of usr intrfcs sird proprtis of systm with finit or infinit stt spc) my provd in smi-utomtd wy givn forml spcifiction or modl of th systm nd forml dscription of thos proprtis [10 5 Excutl spcifictions or modls) of usr intrfcs nd intrctiv systms my usd s fully functionl prototyps Prolms in spcifiction nd dsign cn discovrd nd corrctd for implmnttion gins 6 In rstrictd domins nd with pproprit tool support s for xmpl [11) forml spcifictions or modls of usr intrfcs cn usd s th sis for th utomtic gnrtion of n implmnttion in som trgt pltform ccording to rfinmnt or trnsltion ruls Th gnrtd implmnttions r corrct y construction nd conformity tsts r not ndd Ovrll highr rigor in th dscription nd vrifiction of intrctiv componnts is importnt to gin confidnc on thir corrctnss nd ncourg thir rus [10 EXMPLE In this xmpl th sml Tstr tool is usd to tst th conformity twn th implmnttion of th utton control in th Nt frmwork clss) nd spcifiction of smll prt of its hviour rltd to mous nd kyord vnts only) in th sml lngug Th xmpl is smll ut ws slctd minly to illustrt th tsting procss nd turnrounds to som difficultis nd not th powr of th sml lngug Th pproch prsntd cn sily scl to usd in lrgr intrctiv controls 1 Forml Spcifiction of Button in sml Th utton spcifiction hs instnc vrils Spcifiction 1) vnts Spcifiction 2) nd mthods Spcifiction 3) Two typs of vnts should distinguishd: vnts rcivd y th utton nd ) nd vnts gnrtd y th utton in rspons to th prvious ons ) Both ths typs of vnts my snt y th utton to th ppliction vi vnt hndlrs Thr r mor utton vnts in th Nt pltform ut only ths vnts r considrd in this xmpl Instnc vrils nd Spcifiction 1) modl th intrnl stt of th utton nd corrspond to proprtis xisting in th clss in th Nt frmwork Instnc vrils nd rprsnt vnt flgs tht wr ddd to chck tht pproprit squncs of mous nd kyord vnts r rcivd y utton ftr ftr ) Instnc vril ws ddd to tll whthr th vnt ws gnrtd y th utton in rspons to th lst kyord or mous vnt rcivd This instnc vril is rst ch tim n vnt is rcivd mthod! " ) # & & 8 ' ) 9 ) & - :) < /0 = & ) B 5 2 C & ' ' F C G H B@ = & ' E & IJ G H B@ ' E & & 6 7 < 5 L B@ = & ' E & Spcifiction 1 - Button instnc vrils Evnts r dfind in wy similr to th Nt frmwork Spcifiction 2) For ch vnt nmd M NO P QR S T O thr is n instnc vril nmd M NOP QR S T O U S P V WO XY tht stors th vnt hndlrs rgistrd with th vnt Oprtions nd " r rsponsil to rgistr nd unrgistr vnt hndlrs In n sml spcifiction it is possil to us typs dfind in th Nt frmwork Tht's th cs of th typ " Z in Spcifiction 2 nd typs " [ " [ nd " [ in Spcifiction / 6 < 7 5 K < & 5 L^ ` ) ^ C & : ) _ E ` ) ^ c _ c _ ` _ E 7_ < c 5 L^ < 5 L^ C C < ) ) C <) ) f ` ) C C d d IJ G H g IJ h i g F C G H g F C h i Spcifiction 2 Button vnts For ch vnt nmd M NO P QR S T O thr is mthod nmd j P M NO P QR S T O tht xcuts whn th vnt occurs Th mthod is rsponsil for clling th vnt hndlrs tht wr rgistrd with th vnt Spcifiction 3) Mthods r pulic y dfult In th cs of vnts rcivd y th utton nd ) th rcption of th vnt my simultd for tsting purposs) y clling th mthod j P M NO P QR S T O from th outsid of th clss This xplins why thos mthods r mrkd s pulic In th cs of vnts gnrtd intrnlly y th utton ) th mthod is mrkd s protctd cus it should only clld intrnlly Som of th mthods hv pr-condition indictd with th kyword k Th pr-condition rstricts th stts whr th mthod cn xcut For instnc th mthod l cn only xcut whn th proprty m is tru ftr th pr-condition coms th modl progrm dscrid in sction 3) In som css th modl progrm is orgnizd in stps

5 5 QUTIC 200 PROCEEINGS 7_ # 0 1 < 5 L & ` ) = C 7 E _ / 7 < 5 L^ C c g < 5 L B@ = 7 1 IJ G H & IJ ` ) = C 7 IJ G 7 H B@ 2 = E & 7 / 0 B 5 C 7 /0 & C ) ) ` ) B@ = C & E _ 7 c g / IJ G H ^ C IJ G H B@ = 7 1 IJ h i & IJ ` ) = C 7 IJ G 7 H B@ 2 = 7 7 / 0 B 5 C 7 /0 & C ) ) ` ) B@ = C & E _ 7 / IJ h i^ C c g & E IJ G ) < H C B C IJ C : i 5 / 1 < 5 L / ` ) = C IJ G H B@ = E & 1 F C G H & F C ` ) = C 7 & 7 C ) 2 7 / 0 F C G H B@ = E & ) ` ) B@ = C & E _ 7 c g / F C G H ^ C F B 5 C C G H B@ = F C h i & F C ` ) = C 7 F C 7 G H B@ 2 = 7 7 / 0 B 5 C 7 /0 & C ) ) ` ) B@ = C & E _ 7 c g / F C h i^ C & 1 < 5 L / ` ) = C F C G H B@ = E & & 7 0 ' ) ) C & :) < = g 2 & ' 8 9 ) 2 C 0 0 c _ ' 2 5 C C ) ) ` ) B@ = C < 5 L B@ = E & Spcifiction 3 Button mthods 2 Extnsion of th Button Implmnttion Clss for Tstility In ordr to ovrcom th limitd tstility of th clss in th Nt frmwork Tstl Button) clss inhriting from ws crtd in C# Implmnttion 1) Evnt flg is similr to th on usd in th spcifiction This flg ws mrkd s pulic to fcilitt th mpping of stts twn th spcifiction nd th implmnttion Mthod l ovrrids protctd mthod with th sm nm dfind in th clss in ordr to mnipult th ddd vnt flg nd rcord th occurrnc of th vnt It clls th s mthod so tht th vnt hndlrs r clld i so tht th vnt is snt to th ppliction) For ch vnt rcivd from th usr) with nm M NOP QR S T O it ws ddd pulic mthod nm j P M NOP QR S T O U-Usr) tht cn clld from outsid th clss to simult usr vnts # & & 8 ' ) ) ' ) ) ' < 5 L B@ = E & # 8 ' ) ) :) < = C g' 2 2 C 8 9 ) C 2 2 7_ # 0 _ _ 0 1 < 5 L ` ) = C 2 C 1 < 5 L < 5 L B@ = ) # 6 _ 0 h1 IJ G H IJ ` ) = C < 5 L B@ = C 1 IJ G H C < ) d d IJ h i g F C G H C g <) ) F f C h ` i ) C Implmnttion 1 Implmnttion of tstl utton clss TButton) 3 Spcifiction nd Implmnttion of Tst Continr In ordr for usr to intrct with utton it must md visil y putting it insid som window or continr With this purpos clss ws crtd oth t th spcifiction lvl Spcifiction ) nd t th implmnttion lvl Implmnttion 2) u to limittions of th tst tool uxiliry mthods nd hd to crtd to simult usr vnts tht r snt to th utton contind in th form Ths mthods r slctd to triggr th trnsitions in th stt trnsition digrm Figur 1) Ech tst cs will constructd s squnc of clls to ths mthods # 6 & & 7 8 ' B & ' ) )!" # / & ' ) ) ) 8 ' ) ) g 7 #- / 8 ' 1 F 0 1 C G H! " # & ' ) # - / 8 ' 1 F C h i 8 ' IJ G H & < H C B C IJ ` ) = C 8 ' 1 IJ G H 8 ' IJ h i & < H C B C IJ ` ) = C 8 ' 1 IJ h i & ' B d d < C ) ) ) 2 ) < C < 5 < ) C ) f < 5 C 0 0 c _ < 5 C Spcifiction Continr clss TBForm G H < = & / 2 < H C B C / c & # 8 ' B # # & & 8 ' B 76 8 ' ) ) 8 ' B # 8 ' B <) < 3 K i ) 8 H 8 ' ) ) 8 ' ) ) g) ) < / C 8 ' 5 5 ) < G H < = 6 < ) 7 8 g ' : 8 ' K C ) < C 8 ' ) < C : H d d ) J

6 6 QUTIC 200 PROCEEINGS = < 76 6 < _ 0 H C B G C <= = ) 5 <) < 3 K i ) ) < C ) : 5 ' C : < 3 / G H < = : < 3 g 7 ) < C < ) : < 3 / G H < = : < 3 g : 8 B 8 9 ) 8 ' B ) < C C 5J ) C :8 8 = < & # 6 _ 0 F < i < 5 ) < / 8 ' B # 6 _ 0 8 ' IJ G H < H C B C IJ ` ) = C 8 ' h1 IJ G H 76 6 _ 0 8 ' IJ h i < H C B C IJ ` ) = C 8 ' h1 IJ h i 76 6 _ 0 8 'F C G H < H C B C F C ` ) = C 8 ' h1 F C G H 76 6 _ 0 8 'F C h i < H C B C F C ` ) = C 8 ' h1 F C h i Implmnttion 2 Continr clss TBForm Gnrtion of th Finit Stt Mchin Modl Th sml Tstr tool ws usd to utomticlly gnrt th finit stt mchin FSM) from th sml spcifiction sd on th following configurtion informtion: 1 List of stt vrils - ll th vnt flgs dfind in th spcifiction of th clss m nd ) 2 List of ctions tht triggr trnsitions th constructor nd mthods dfind in th clss nd m ) 3 omins vlus to considr) for th prvious stt vrils nd ctions rgumnts For ch stt vril th domin is In th cs of th mous ctions singl rgumnt ws providd spcifying th lft mous utton nd position insid th form utton mor prcisly th vlu " [ In th cs of th kyord ctions singl rgumnt ws providd spcifying th ky tht is th vlu " [ [ Th FSM otind is shown in Figur 1 Ech stt corrsponds to comintion of vlus of th stt vrils prt from th strt stt gryd) th FSM hs 6 stts This mns tht two of th possil comintions of vlus of th thr stt vrils cnnot occur Th possil trnsitions dprting from ch stt r constrind y th mthods pr-conditions k clus) Figur 1 Finit Stt Mchin FSM) digrm 5 finition of Mppings twn th Spcifiction nd th Implmnttion To prform conformity tsts it is ncssry to dfin mppings conformnc rltions) twn spcifiction nd implmnttion mthods nd dt stt) [8 Ths rltions cn stlishd mnully or in n utomtd wy It ws dfind rltion twn th clsss spcifiction) nd [ m implmnttion) ftr this th mthods with th sm nms nd rgumnts in oth clsss r utomticlly rltd In th currnt vrsion of th tst tool dt rltions hv to dfind mnully In this xmpl only th instnc vril ws mppd s shown in Figur 2 Conformnc tsts will xcut rltd mthods in oth lvls spcifiction nd implmnttion) nd will compr rsults otind from oth nd lso compr th rltd dt Figur 2 Conformnc rltions configurtion 6 Gnrtion of th Tst Suit ftr gnrting th FSM it is possil to gnrt tst suit utomticlly Figur 3) sd on covrg critri slctd y th usr In this cs it ws usd th dfult critri tht nsurs full covrg of stts nd trnsitions tst suit with singl tst cs squnc) ws sufficint to covr ll th trnsitions This tst suit will usd s th input to conformnc tsting

7 7 QUTIC 200 PROCEEINGS spcr ky Figur 3 Tst suit gnrtd 7 Tst Excution nd Rsults s soon s th conformity rltions r dfind nd th FSM nd th tst suit r gnrtd it is possil to xcut conformnc tsts Evry tim thr is n inconsistnc th tool stops nd rports th rror Th tool rports conformnc rror whn th squnc of vnts nd is xcutd Figur ) with ky '' Th rror is n inconsistncy twn th vlu of th vlu t th implmnttion th vlu is tru) nd th spcifiction th vlu is fls) This mns tht th implmnttion th clss in th Nt frmwork) gnrts vnt whn it rcivs from th usr th squnc of vnts nd ccording to th documnttion of th Nt frmwork this should only hppn whn th ky prssd is th spcr which is not th cs hr) Error Figur Conformnc tst inconsistncy th pth to th rror is shown in rd) To rproduc this norml hviour mnully it is ncssry to prss th lft mous utton on Nt utton nd prss nd rls kyord ky without rlsing th mous utton This will hv th ffct of slcting th utton nd xcuting th ction ssocitd with it ccording to th documnttion this should only hppn with th 5 CONCLUSION n pproch to tst intrctiv componnts with th utomtic gnrtion of tst css from spcifiction ws dscrid In comprison with othrs th pproch prsntd in this ppr rquirs forml spcifiction with dmonstrtd nfits in th dvlopmnt nd vrifiction of intrctiv componnts In th pst forml spcifiction nd vrifiction tchniqus hv n usd minly in th dvlopmnt of criticl systms ut from our point of viw thy lso hv mjor rol to ply in th dvlopmnt nd vrifiction of rusl componnts s is th cs of intrctiv componnts It ws prsntd n xmpl of utomtic tsting th conformity twn th implmnttion of utton in th Nt frmwork nd spcifiction writtn in th sml lngug using th sml Tstr tool Som tst cod ws ndd to ovrcom tstility limittions of th trgt cod lthough only smll prt of th hviour of utton ws spcifid nd tstd th tsts wr succssful tht is ug ws dtctd lrgr xmpl could usd sinc th pproch cn sily scl ut it would difficult to xplin tht xmpl in fw pgs Howvr in its currnt stt th sml Tstr tool lso hs som limittions: 1 It still rquirs too much usr intrvntion 2 Whil th tight intgrtion with th Nt frmwork hs som dvntgs on of its shortcomings riss from th fct tht th lvl of strction of th spcifiction is not s high s should 3 Intrctiv componnts cn hv lots of stts nd ctions or vnts tht cn hrd to mnipult nd tst Th sml Tstr tool llows th slction of which ctions should ppr in th FSM digrm nd in th tst css gnrtd from th FSM) Consquntly it is possil to tst sprtly prts of th hviour of th ojct or componnt undr tst But rigorous mthod is ndd to dfin thos prts nd sum th rsults otind in ch prt to tk covrg critri conclusions Th pproch prsntd in this ppr hs to xtndd nd mturd in svrl dirctions: 1 Us th pproch prsntd in lrgr xmpls 2 Explor othr wys to gnrt tst css from th FSM modl som critri to gnrt spcifictionsd tsts cn found t [12 3 fin dditionl chck points for instnc whn cllck is issud nd on rturn Modl Chcking intgrt th pproch with modl chcking tchniqus to prov proprtis out th modl 5 Vrifiction of th usr intrfc contrct prticulrly chllnging is th prolm of chcking tht th outputs snt y n intrctiv componnt to th usr oy to som kind of spcifiction or contrct For xmpl th usr intrfc contrct of txtox is to llow th usr to insrt nd visuliz string through smll

8 8 QUTIC 200 PROCEEINGS window ov points nd possil othrs will sujct of futur work CKNOWLEGMENT Th uthors wish to thnk th nonymous rviwrs for thir commnts nd suggstions REFERENCES [1 R V Bindr Tsting Ojct-Orintd Systms: Modls Pttrns nd Tools: ddison-wsly 2000 [2 J Z Go H-S J Tso nd Y Wu Tsting nd Qulity ssurnc for Componnt-Bsd Softwr: rtch Hous Pulishrs 2003 [3 C Szyprski Componnt Softwr: Byond Ojct-Orintd Progrmming: ddison-wsly 1999 [ C Knr J Bch nd B Pttichord Lssons Lrnd in Softwr Tsting: Contxt-rivn pproch: John Wily & Sons 2002 [5 B Myr "pplying sign y Contrct" IEEE Computr pp [6 F Findlr "Contrct Soundnss for Ojct-Orintd Lngugs" prsntd t Ojct-Orintd Progrmming Systms Lngugs nd pplictions OOPSL) 2001 [7 Microsoft "Introducing sml: Tutoril for th strct Stt Mchin Lngug" Foundtions of Softwr Rsrch 2002 [8 C Piv J P Fri nd R M Vidl "Spcifiction-sd Tsting of Usr Intrfcs" prsntd t 10th SV-IS Workshop - sign Spcifiction nd Vrifiction of Intrctiv Systms Funchl - Mdir 2003 [9 J Cmpos nd M Hrrison "Modl Chcking Intrctor Spcifictions" in utomtd Softwr Enginring vol [10 I McColl nd Crrington "Usr Intrfc Corrctnss" prsntd t Humn Computr Intrction - HCI' [11 M Lozno "Entorno Mtodológico Orintdo Ojctos pr l Espcificción y srrollo d Intrfcs d Usurio" in Sistms Informáticos y Computción Vlnci: Univrsidd Politécnic d Vlnci 2001 [12 J Offutt S Liu durzik nd P mmnn "Gnrting tst dt from stt-sd spcifictions" Softwr Tsting Vrifiction nd Rliility vol 13 pp n C R Piv rcivd MSc dgr in Elctricl nd Computrs Enginring from Enginring Fculty of Porto Univrsity FEUP) nd dgr in Informtion Systms Enginring from Minho Univrsity of Portugl in 1997 nd 1995 rspctivly Sh is currntly dvloping hrs doctort in forml mthods pplid to usr intrfcs t FEUP Elctricl nd Computrs Enginring prtmnt whr sh is n ssistnt Lctur sinc 1999 João C P Fri rcivd Ph in Elctricl nd Computr Enginring from th Enginring Fculty of Porto Univrsity FEUP) in 1999 nd dgr in Elctricl Enginring from FEUP in 1985 H is n ssistnt Profssor t FEUP Elctricl nd Computrs Enginring prtmnt Informtics Rul F M Vidl rcivd Ph in igitl Elctronics t UMIST in 1978 n MSc in Communiction Enginring t UMIST in 197 nd dgr in Elctricl Enginring t Enginring Fculty of Porto Univrsity FEUP) in 1972 H is n ssocit Profssor t FEUP Elctricl nd Computrs Enginring prtmnt Informtics