Insiue for Sofware Technology Qualiy Assurance in Sofware Developmen Qualiässicherung in der Sofwareenwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Insiue for Sofware Technology Graz Universiy of Technology Ausria Summer Term 2015 1 / 31
Agenda 1 Conformance Tesing Properies 2 Labelled Transiion Sysems Equivalence Preorder ioco Examples 3 Tes generaion 2 / 31
Insiue for Sofware Technology Inpu/Oupu Conformance Relaion Jan Tremans - 1996 Based on IO-Labeled Transiion Sysems 3 / 31
Insiue for Sofware Technology Inpu/Oupu Conformance Relaion Jan Tremans - 1996 Based on IO-Labeled Transiion Sysems 3 / 31
Conformance Tesing 4 / 31
Conformance Tesing - Soundness Tes suie is sound: conformance all ess pass 5 / 31
Conformance Tesing - Soundness Tes suie is sound: conformance all ess pass 5 / 31
Conformance Tesing - Exhausiveness Tes suie is exhausive: conformance all ess pass 6 / 31
Conformance Tesing - Exhausiveness Tes suie is exhausive: conformance all ess pass 6 / 31
Conformance Tesing - Compleeness Tes suie is complee: conformance all ess pass 7 / 31
Conformance Tesing - Compleeness Tes suie is complee: conformance all ess pass 7 / 31
Conformance Tesing wih ioco Sysem (Implemenaion) is modeled as IOTS weakly inpu enabled Specificaion is an IOLTS possibly incomplee possible non-deerminisic!g1 τ τ!g1?g2!g3?g2!g3!g1 8 / 31
Conformance Tesing wih ioco Sysem (Implemenaion) is modeled as IOTS weakly inpu enabled Specificaion is an IOLTS possibly incomplee possible non-deerminisic!g1 τ τ!g1?g2!g3?g2!g3!g1 8 / 31
Inpu Oupu Labeled Transiion Sysems Inpu Oupu Labeled Transiion Sysem An IOLTS is an LTS M = (Q M, A M, M, q M 0 ) wih Q M a finie se of saes A M = A M I A M O {τ} where A M I and A M O are inpu and oupu alphabes τ A M I A M O is an unobservable, inernal acion M Q M A M Q M is he ransiion relaion q M 0 QUESTION? QM is he iniial sae. Wha means: The implemenaion conforms-o he specificaion? 9 / 31
Inpu Oupu Labeled Transiion Sysems Inpu Oupu Labeled Transiion Sysem An IOLTS is an LTS M = (Q M, A M, M, q M 0 ) wih Q M a finie se of saes A M = A M I A M O {τ} where A M I and A M O are inpu and oupu alphabes τ A M I A M O is an unobservable, inernal acion M Q M A M Q M is he ransiion relaion q M 0 QUESTION? QM is he iniial sae. Wha means: The implemenaion conforms-o he specificaion? 9 / 31
How o relae 2 LTSs? Equivalence Relaions (=) Bisimulaion Trace Equivalence Tesing Equivalence... Preorder Relaions ( ) Trace Preorder Tesing Preorder... Inpu-Oupu Relaions ioconf ioco...... 10 / 31
(Weak) Bisimulaion Two saes are bisimilar iff hey simulae each oher and go o saes which are bisimilar Bisimulaion is no suied for esing! 11 / 31
Insiue for Sofware Technology Trace Equivalence A race is an observable sequence of acions Two saes are race equivalen iff hey have he same races Trace equivalence is he weakes noion of conformance 12 / 31
Equivalence vs. Preorder Relaions Equivalence Relaion (R) reflexive (srs) symmeric: irs sri ransiive: irs sr ir Preorder Relaions ( ) NOT necessarily anisymmeric: irs i s s i simplifies esing e.g.: Trace Preorder i r s races(i) races(s) 13 / 31
Insiue for Sofware Technology Some Noaions: Transiions q a M q = df (q, a, q ) M q ɛ q = df (q = q ) (q τ M q 1 q n 1 τ M q ) q a q = df q 1, q 2 : q ɛ M q 1 a M q 2 ɛ M q 14 / 31
Some Noaions: Quiescence δ is used o represen quiescence q δ q = df q is a quiescen sae. Quiescen sae = no edge labeled wih an oupu or an inernal acion 15 / 31
Some Noaions: Quiescence δ is used o represen quiescence q δ q = df q is a quiescen sae. Quiescen sae = no edge labeled wih an oupu or an inernal acion 15 / 31
Some Noaions: Suspension Auomaon (M) = (Q M, A (M), (M), q0 M) where: A (M) = A M {δ} wih δ A (M) O (M) is obained from M by adding loops q δ q for each quiescen sae 16 / 31
Insiue for Sofware Technology Some Noaions: Afer q afer M σ = df {q q σ M q } Q afer M σ = df q Q (q afer M σ). 17 / 31
Insiue for Sofware Technology Some Noaions: Ou Ou M (q) = df {a A M O q a M } Ou M (Q) = df q Q (Ou M(q)) 18 / 31
ioco Definiion: ioco Le IUT = (Q IUT, A IUT, IUT, q0 IUT ) be weakly inpu enabled wih A IUT = A IUT I A IUT O {τ} and S = ( ) Q S, A S, S, q0 S be srongly responsive wih A S = A S I A S O {τ}. Then: IUT ioco S = df σ races( (S)) : Ou IUT ( (IUT ) afer IUT σ) Ou S ( (S) afer S σ). IUT ioco S iff oupus (and quiescences) of he IUT are possible in S afer an arbirary suspension race of S. 19 / 31
ioco Definiion: ioco Le IUT = (Q IUT, A IUT, IUT, q0 IUT ) be weakly inpu enabled wih A IUT = A IUT I A IUT O {τ} and S = ( ) Q S, A S, S, q0 S be srongly responsive wih A S = A S I A S O {τ}. Then: IUT ioco S = df σ races( (S)) : Ou IUT ( (IUT ) afer IUT σ) Ou S ( (S) afer S σ). IUT ioco S iff oupus (and quiescences) of he IUT are possible in S afer an arbirary suspension race of S. 19 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 20 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 20 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 21 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 21 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 22 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 22 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 23 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 23 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 24 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 24 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 25 / 31
Insiue for Sofware Technology P ioco S? P ioco S = df σ races( (S)) : Ou IUT ( (P) afer IUT σ) Ou S ( (S) afer S σ). 25 / 31
Tes Cases A es case is an IOLTS Inpus = Oupus IUT, Oupus = Inpus IUT Equipped wih verdic saes (pass, fail) In each sae (excep Pass, Fail): Single oupu and all inpus All inpus and θ 26 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
Formal Tes Execuion 27 / 31
A Complee Tes Generaion Algorihm Given he suspension auomaon of a specificaion as an LTS S = (Q S, A S, S, q S 0 ) 1 Iniially compue K = q0 S afer S ɛ 2 Do non-deerminisically, eiher: Sop es case wih verdic pass Le he es case produce an oupu (!a) wih K = K afer S?a. Also accep all inpus a he same ime and add fail saes for unexpeced resuls. Accep all inpus (and quiescence) and add fail saes for unexpeced resuls. Compue new K for valid inpus. 3 Repea Sep 2 wih new se K. 28 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
A Complee Tes Generaion Algorihm δ δ 29 / 31
Insiue for Sofware Technology Tools TGV: offline esing ool jtorx: online esing MoMu: offline model-based muaion esing ool, AIT and TU Graz SpecExplorer: uses Alernaing Simulaion (equivalen o ioco for deerminisic LTS) 30 / 31
References Marin Weiglhofer, Bernhard Aichernig, and Franz Woawa. Faul-based conformance esing in pracice. Inernaional Journal of Sofware and Informaics, 3(2-3):375 411, June/Sepember 2009. Copyrigh by Insiue for Sofware, Chinese Academy of Science. 31 / 31