Automatic Testing of Neighbo Discovey Potocol Based on FSM and TTCN* Zhiliang Wang, Xia Yin, Haibin Wang, and Jianping Wu Depatment of Compute Science, Tsinghua Univesity Beijing, P. R. China, 100084 Email: {wzl, yxia, wanghb}@csnet1.cs.tsinghua.edu.cn, jianping@cenet.edu.cn Abstact Potocol confomance testing is an impotant method to ensue the quality of netwok communication softwae. In this pape, we pesent a fomal method to test neighbo discovey potocol, which is one of the basic IPv6 potocols. The fomal model EFSM is extended with simple timed opeations to specify potocol behavios. We popose a pactical adjusted test geneation method applied to such a model. We also implement a test suite in TTCN and pefom automatic testing to IPv6 outes in ou test system PITS. Test esults povide helpful efeence to the development of IPv6 oute. Keywods Neighbo Discovey Potocol(NDP), EFSM, Confomance Testing, TTCN, Test System, IPv6 I. INTRODUCTION To ovecome the addess space explosion poblem, a new vesion of Intenet Potocol IPv6 ([8]) has been poposed to substitute the cuent vesion IPv4. Up to now, moe and moe netwok devices suppoting IPv6 have emeged. Neighbo Discovey Potocol ([9]) is one of the basic IPv6 potocols, which should be suppoted in all IPv6 implementations. Potocol test techniques ae widely used in ode to ensue the quality of netwok communication softwae. Confomance testing ([2]) is a basic method of potocol testing, which can be used to test whethe an implementation confoms to its potocol specification. Many eseaches have been done on neighbo discovey potocol testing, such as [1] [10] [11]. In all these woks, few fomal methods have been used: [1] specifies test cases in natual language and its test suite is not complete towads the potocol specification; [10] specifies test cases in Pel scipt, which ae difficult to develop and undestand; [11] poposed a test specification language called TSS to specify test cases, but TSS is not a standad language and has poo scalability. Futhemoe, they all should be executed on the popietay test systems. On the othe hand, in all these woks, test cases ae developed manually accoding to the textual potocol specifications--rfc, which is not guided by a fomal method. Such a way cannot ensue test coveage and eliability of test suite. In this pape, we popose a fomal method to test neighbo discovey potocol. In the method, FSM (Finite State Machine)-based fomal technique is used to specify the potocol unde test and TTCN (Tee and Tabula Notations [12]) is used to specify test cases. Such techniques ensue test coveage to the potocol specification and make the whole test pocess automatic. Test pactice has been pefomed on ou univesal TTCN-based test system PITS ([13]), on which many test activities on IPv6 potocol suite have been done in ou pevious wok ([6] [14]). The emainde of this pape is stuctued as follows. Section II intoduces neighbo discovey potocol and the fomal model we use to specify it. Section III pesents a test geneation method applied on the fomal model. In section IV, techniques used in pactical test activities ae given, and we also illustate the test esults to eal-life oute implementations. Conclusion and futue wok ae given in section V. II. PROTOCOL SPECIFICATION AND FORMAL MODEL A. Neighbo Discoveey Potocol Neighbo Discovey (fo shot, ND) Potocol ([9]) is one of the basic potocols in IPv6, which coesponds to a combination of ARP potocol, ICMP oute discovey and ICMP ediect function in IPv4. In addition, Neighbo Uneachability Detection mechanism is supplied to enhance the obustness of packet tansmission. All functions in ND potocol ae implemented by using five ICMP packet types, a pai of Route Solicitation and Route Advetisement messages, a pai of Neighbo Solicitation and Neighbo Advetisements messages, and a Rediect message, unifomly. In ND potocol, two types of nodes exist in a link: oute and host. Route is the node that fowads IP packets not explicitly addessed to itself, which is the elay node in the link; while host is any node that is not a oute, which is the end node. B. Fomal Model Neighbo Uneachability Detection (fo shot, NUD) is an impotant function in NDP. We popose a fomal model of Extended Finite State Machine (EFSM) with Simple Timed Extensions to specify the potocol behavios. [Definition 1] EFSM with Simple Timed Extensions An EFSM with Simple Timed Extensions is a 6-tuple M={S, I, O, X, T, s 0 }, whee S, I, O, X, T ae finite sets of states, *Suppoted by the National Natual Science Foundation of China unde Gant No. 90104002, the National High Technology Development 863 pogam of China unde Gant No. 2001AA112061, and 973 Pogam of China unde Gant No. 2003CB314801.
input symbols, output symbols, context vaiables and tansitions espectively. Each state has a local time Ts, which is available only in state s S; Each tansition t T is an 8-tuple t={s,q,i,o,p,op,up,tact}, which is denoted as: i[ P] /{ tact o s op ; up ; } q, whee: (1) s, q S ae the initial and final states of t; (2) i I is an input symbol, p i D( p i ) is the paamete vecto of input i, whee D( p i ) is the set of possible input paamete vectos associated with the input i; Timeout of the local time Ts is a special type of input, which is denoted as timeout(ts), such a tansition is so-called timeout tansition; and o O is an output symbol, p o D( p o ) is the paamete vecto of output o, whee D( p o ) is the set of possible output paamete vectos associated with the output o; (3) P: D( p i ) D( { Tue, False} is a pedicate, whee x is a context vaiable vecto and D(x ) is a set of context vectos; If the tansition has no pedicate, P={Tue}; op: D( pi ) D( D( po ) is an output paamete function; up: D( p i ) D( D( is a context vaiable update function; (4) tact is the timed action to invoke the local time Tq, which can be denoted as stat(tq); If tact is NULL, thee is no timed action fo this tansition. In this model, timed behavios ae simple: thee is only one unning time at most in a state, and expiation of such a local time will invoke a timeout tansition. Such a model can specify potocol behavios of Neighbo Uneachability Detection conveniently (see Appendix A). In a node, each machine tacks the eachability "state" of a Neighbo Cache enty, to which it is sending packets, and ecods the link laye addess of the neighbo. State NONE epesents that thee exists no cache enty to the coesponding neighbo in the node. In this model, an ICMP Echo Request packet is used to invoke the sending of fowading taffic (ICMP Echo Reply) to the neighbo, which is a easonable simplification to the potocol specification fom the testing pespective. [Definition 2] Extenal Obsevable timed featues Fo an EFSM with Simple Timed Extensions M, its cuent state is s S. We do not apply any extenal inputs to it. If an extenal output o can be obseved afte waiting a while, we say that state s has an Extenal Obsevable Timed Featue and its obseved output is o. In fact, such featues can be consideed as the extenal behavios of timeout tansitions. We illustate it with the example shown in Appendix A: table I is a summay of such featues fo all states. In this model, the states with this featue ae all tentative ones. In Section III, such a featue can be helpful to geneate adaptive distinguishing sequence. TABLE I. EXTERNAL OBSERVABLE TIMED FEATURES OF NUD FSM State Has such a featue? Obseved output time(s) NONE N -- -- INCOMPLETE Y mns (0,1) STALE N -- -- REACHABLE N -- -- DELAY Y (0,5) PROBE Y (0,1) III. TEST GENERATION A. Test Achitectue In confomance testing, a potocol implementation can be consideed as a black-box, so we can apply inputs to it and obseve its outputs to see if it confoms to its specification. In IUT (Implementation Unde Test), the timed pat is used to contol its timed behavio. But this pat cannot be contolled and obseved extenally, that is, teste has no capability to detemine when a time is stated o timed-out. Fig. 1 shows a kind of test achitectue fo such an IUT. Test system communicates with IUT via a PCO (Point of Contol and Obsevation), and a time module is included to help testing timed behavios of IUT, but no access points can each IUT s timed pat. Test System PCO IUT time timed pat Figue 1. test achitetue fo ND potocol B. Test Geneation In [4], confomance testing based on FSM has been suveyed. Geneally speaking, confomance test suite to an FSM consists of two pats: (1) state cove TS 1 and (2) tansition cove TS 2. [Definition 3] Peambles and Distinguishing Sequences Fo one state s S, (1) a peamble pe(s) leads the machine fom the initial state s 0 to the state s; (2) a distinguishing sequence ds(s) can be used to veify whethe the cuent state is s. In the state cove TS 1, fo each state, test case tc_1_i = pe(s i ) ds(s i ); and in the tansition cove TS 2, fo each ti tansition t i : s q, test case tc_2_i = pe(s) t i ds(q). In the beginning of each test case, is a eset opeation to lead the machine to its initial state s 0 (we popose an implementation of in pactical testing pocess in section IV). To cove the state s i, fo the test case tc_1_i, a peamble pe(s i ) fistly leads the machine to the state s i, then ds(s i ) is applied to veify the state s i. To cove the tansition t i, fo the test case tc_2_i, a peamble pe(s) fistly
leads the machine to the state s, which is the initial state of tansition t i ; then the input of t i is applied to the machine to see if it poduces the coect output; afte that, ds(q) is applied to veify the final state q of tansition t i. To constuct the peamble of state s, a path should be geneated fom the initial state s 0 to the state s; many existing algoithms, such as shotest path tee, can be used to solve the poblem easily. To constuct distinguishing sequences, one way is to constuct adaptive distinguishing sequence (fo shot, ADS) ([3]) fistly. In fact, ADS is a decision tee with exactly n leaf nodes, whee n is the numbe of states. The intenal nodes epesent the set of possible cuent states: oot node of the tee is the set of all states. To distinguish the states fom each othe, an input should be applied to the machine, and the edges of the tee label with the diffeent output espond to it. If the set of possible cuent state becomes a singleton {s}, the node will be a leaf, which means that the machine stays exactly in the state s. In ou model, extenal obsevable timed featues ae used to assist geneating ADS. We adjust the algoithms in [3] to ou model: when expanding an intenal node of the decision tee, extenal obsevable timed featues should be fistly consideed; accoding to the diffeent obseved outputs, possible cuent states set can be divided into seveal subsets; then fo the nodes epesenting subsets of states which have the same obseved outputs, we expand them by applying an input as the nomal expanding pocess. ADS of potocol specification in Appendix A is shown in fig. 2. mns (1s) {N,I,S,R,D,P} (1s) (5s) I P D -- mns N {N,S,R} EReq (5s) S ERep {S,R} Figue 2. adaptive distinguishing sequence of model in Appendix A Fom the ADS, it is easy to constuct distinguishing sequence fo some state s i, which is the path fom the oot node to the leaf node epesenting state s i. TABLE II. PREAMBLES AND DISTINGUISHING SEQS FOR ALL STATES State pe(s i ) Distinguishing seq. (ds(s i )) N -- /--;?EReq/!mNS I t 1 (1s)/!mNS S t 1 t 4 /--;?EReq/!EPep; (5s)/! R t 1 t 5 /--;?EReq/!EPep; /-- D t 1 t 4 t 6 (5s)/! P t 1 t 4 t 6 t 7 (1s)/! -- R Table. II shows the peambles and distinguishing sequences fo all states, which can be combined into the whole test suite. Afte geneating skeletons of test cases, a futhe step should be taken to ealize TTCN test cases accoding to them. IV. TEST PRACTICE A. Test System To pefom pactical test activities, a potocol test system based on TTCN-2 -- PITS (Potocol Integated Test System) [13] can be used. Fig. 3 shows a kind of typical test achitectue which is implemented in PITS. Test Execution is the main engine of the PITS, which eads the TTCN test cases and executes them in an intepetive way. We implement an RI (Refeence Implementation) to communicate with IUT. Following the desciption of test cases, Test Execution contols RI to send (eceive) potocol messages to (fom) PCO. In PITS, a time module is also used to ealize the time opeations in TTCN test cases. Afte the execution of a test case, Test Execution will give a vedict of PASS, FAIL o INCONCLUSIVE accoding to the test esult. Figue 3. Pactical Test Achitectue in PITS B. Function of RI In PITS, RI ([5]) is the potocol dependent module to inteact with IUT. Its basic function is to send o eceive potocol messages. In ND potocol testing, many test cases equie diffeent configuations of IUT. Befoe execution of each test case, IUT should be configued popely accoding to the equiement of the test case. Anothe equiement of test configuation is the implementation of the eset opeation, which leads the state machine of the IUT to its initial state NONE, i.e., deletes the coesponding neighbo cache enty. Such a eset opeation should be used in the beginning of each test case. So the above two equiements of test configuation can be implemented in a unifom way, such as telnet. all_conf.dat $IUT_ConfigInfo $IUT_SYS_NAME %IUT_sys $TELNET_IF_IPADDR %ip_conf $TELNET_USERNAME %usename $TELNET_PASSWD %passwd $End_ IUT_ConfigInfo $TestCaseConfig $TestGoupId %testgoup_name $TestCaseId %testcase_name %conf_filename $End_TestCaseConfig Test Suite Configuation File %IUT_sys/%conf_filename clea ipv6 neighbos Test Configuation Scipt (cisco/clea_nc_item) Figue 4. Machinasm of automatic configuation
In ode to make test pocess fully automatic, test configuation should also be automatic. We extend the function of RI to suppot automatic configuation of IUT. The mechanism of such a function is shown in fig. 4: A test suite configuation desciption file (all_conf.dat) is used to specify the configuation infomation of all test cases. Since automatic configuation is ealized by telnet, some necessay infomation about telnet should be specified (in line 3~5). To ealize automatic configuation in diffeent type of IUTs, system type name of IUT should be specified in line 2. Fo each test case in test suite, we use a configuation scipt file to specify its configuation. A mapping will be made fom %IUT_sys and %testcase_name to the configuation scipt file name %IUT_sys/%conf_filename. Fig. 4 shows a scipt file to clea neighbo cache item fo Cisco oute, which acts as a eset opeation. Befoe the execution of a test case, RI eads the test suite configuation file to get the configuation scipt file fo the cuent test case, and pefoms emote configuation in IUT via telnet potocol automatically. The functional model of RI is shown as fig. 5. Test Execution Massage Pocessing Automatic Configuation Telnet Main Thead Test System Packet Sending Lowe Laye Potocol Sevices IUT Packet Receiving Thead Figue 5. Functional Model of RI C. Test Results and Analysis We have implemented a TTCN ND potocol test suite fo oute node. The test suite is composed of 3 test goups (see Appendix B): Route_Discovey, Addess_Resolution and NUD_FSM. The total numbe of test cases is 91. Using the ND potocol test suite, we have tested 3 kinds of oute implementations: oute poduct A and B, and ou oute pototype (I fo the 1st time, II fo the 2nd time). Fig. 6 and Table. III show the test esults, which ae epesented as pass ate. Test esults of Pototype I have been used to fix the poblems found in ou oute pototype. Test esults of the modified pototype ae shown as Pototype II. TABLE III. RI TEST RESULTS STATISTICS Implementation Pass Numbe Pass Rate(%) Poduct A 85 93.4 Poduct B 78 85.7 Pototype I 66 72.5 Pototype II 89 97.8 Statistics and compaison of pass ate 100.0 80.0 60.0 40.0 20.0 0.0 93.4 97.8 85.7 72.5 %Poduct A oute implementations Poduct B Pototype I Pototype II Figue 6. Test Results of oute implementations Fom the esults, we can see that with the help of ND potocol confomance testing, afte modification, ou oute pototype has been impoved geatly: pass ate inceased fom 72.5% to 97.8%, which is highe than the two oute poducts. V. CONCLUSION, DISCUSSION AND FUTURE WORK In this pape, a fomal method has been poposed to test neighbo discovey potocol. A fomal model of EFSM with Simple Timed Extensions is used to specify potocol behavios. Based on such a model, a pactical test geneation method has been pesented. We ealize ND potocol test suite in TTCN-2, which can be executed in ou test system PITS. Finally, we give test esults on seveal oute implementations, which povide a geat deal of help on the eseach and development of ou homemade highend outes. We compae the poposed fomal model in this pape with a geneic model of timed system Timed Input Output Automaton (TIOA) [7]. Fo the potocol behavios with simple timed opeations like Neighbo Uneachability Detection, the method of geneating tests pesented in [7] has temendous costs on time and space, which is difficult to be applied in pactical confomance testing; while ou extended model gives a simple and feasible solution on pactical test activities. In this pape, only testing of potocol contol pat is consideed. Data flow testing fo such a model can be a futue diection. In ou futue wok, futhe test activities on IPv6 potocols, especially outing potocols, should also be done. REFERENCES [1] InteOpeability Laboatoy, Univesity of New Hampshie. "IPv6 Consotium Test Suite: IPv6 Coe Opeations Test Suite". Technical Document. Revision 1.3. Ma. 2003. [2] ISO/IEC 9646. Infomation Pocessing Systems, Open System Inteconnection, OSI Confomance Testing Methodology and Famewok. 1991. [3] Lee, D., Yannakakis, M., "Testing finite state machines: state identification and veification", IEEE Tans. Computes, vol. 43 no. 3, pp. 306-320, 1994. [4] Lee, D., Yannakakis, M., "Pinciples and methods of testing finite state machines-a suvey", Poceedings of the IEEE, Volume: 84, Issue: 8, Aug. 1996. Page(s): 1090 1123.
[5] Zhongjie Li, Jianping Wu, Xia Yin. "Geneality Design fo testing OSI and Intenet Potocols". APCC'2003. Penang, Malaysia, Sep.21-24, 2003. [6] Liu Hong. "Testing of Coe IPv6 Potocols"(In Chinese). Maste Thesis. Tsinghua Univesity. June, 2003. [7] En-Nouaay, A., Dssouli, R., Khendek, F.. Timed Wp-method: testing eal-time systems. IEEE Tansactions on Softwae Engineeing, Volume: 28 Issue: 11, Nov. 2002, pp1023-1038. [8] Deeing S, Hinden R. RFC2460: Intenet Potocol, Vesion 6 (IPv6) Specification. IETF. Decembe 1998. [9] T. Naten, E. Nodmak, W. Simpson. RFC 2461: Neighbo Discovey fo IP Vesion 6 (IPv6). IETF. Decembe 1998. [10] TAHI poject: http://www.tahi.og. [11] Tian Jun et al. "a Fomal Test Method fo Neighbo Discovey Potocol"(In Chinese). Jounal of Compute Reseach and Development, Vol.38, No.12, Dec 2001. [12] ETSI TR 101 666: Infomation Technology - Open Systems Inteconnection, Confomance Testing Methodology and Famewok, Pat 3: Tee and Tabula Combined Notation(TTCN) (Ed. 2++), 1999. [13] Wu Jianping et al, "PITS-The potocol integated test system based on fomal technology"(in Chinese). Jounal of Tsinghua Univesity, 1998, 38(S1):26~29. [14] Ling Zhou, Jianping Wu, Xia Yin, Xingang Shi, Zhiliang Wang. "Reseach and Pactice on Testing Tunnel Potocol". APCC'2003. Penang, Malaysia, Sep.21-24, 2003. t 34 NONE t 1 APPENDIX A PROTOCOL SPECIFICATION OF NEIGHBOR UNREACHABILITY DETECTION t 9 t 36 t 3 t 2 INCOMPLETE t 33 PROBE t 8 t 29 t 5 t 4 t 37 t t 17 t 21 t 39 35 t 7 t 11 t 10 t 12 t 13 t 14 t 15 STALE REACHABLE t t 19 t 20 41 t 40 t 31 t 32 t 28 t 30 t 45 t 22 t 24 t 16 t 18 t 43 t 27 DELAY t 23 t 25 t 26 t 38 t 42 t 44 Tans Potocol Behavio Desciption Tans Potocol Behavio Desciption t1?ereq/{counte:=1;stat(t I)}!mNS t13,t25,t31?na[na.s=0;na.o=0]/-- t2 timeout(t I)[counte<3]/{counte:=counte+1;stat(T I)}!mNS t19?na[na.s=0;na.o=0]/{stat(t R)} t3 timeout(t I)[counte=3]/{counte:=0} t14,t26,t32?na[na.s=0;na.o=1;na.tlla=lla]/-- t4?na[na.s=0]/{lla:=na.tlla}!erep t20?na[na.s=0;na.o=1;na.tlla=lla]/{stat(t R)} t5?na[na.s=1]/{lla:=na.tlla;stat(t R)}!ERep t15,t21,t27,t33?na[na.s=0;na.o=1;na.tlla!=lla]/{lla:=na.tlla} t6?ereq/{stat(t D)}!ERep t34?na/-- t7 timeout(t D)/{counte:=1;stat(T P)}! t35 timeout(t R)/-- t8 timeout(t P)[counte<3]/{counte:=counte+1;stat(T P)}! t36?other/{lla:=other.slla} t9 timeout(t P)[counte=3]/{counte:=0} t37?other/{lla:=other.slla}!erep t10,t16,t22,t28?na[na.s=1;na.o=0;na.tlla=lla]/{stat(t R)} t38,t39,t40,t41?other[other.slla!=lla]/{lla:=other.slla} t11,t17,t23,t29?na[na.s=1;na.o=0;na.tlla!=lla]/-- t42,t43,t44,t45?other[other.slla=lla]/-- t12,t18,t24,t30?na[na.s=1;na.o=1]/{lla:=na.tlla;stat(t R)} APPENDIX B STRUCTURE OF ND PROTOCOL TEST SUITE FOR ROUTER Test Goup Test Pupose Test Cases Num. Basic Basic Function of Route Discovey 9 Route_Discovey NUD_Recv_RS Test NUD FSM: when eceiving an RS 10 Addess_Resolution Basic Function of Addess Resolution 22 NUD_FSM t 6 1. IO symbol: (a) EReq: Echo Request; (b) mns: multicast Neighbo Solicitation; (c) : unicast Neighbo Solicitation; (d) NA(S, O, TLLA): Neighbo Advetisement; 3 input paametes: S is Solicited flag; O is Oveide flag; TLLA is the taget link laye addess; (e) ERep: Echo Reply; (f) OTHER(SLLA): othe ND packet othe than NA. Fo oute, such packets include NS (Neighbo Solication), RS (Route Solication); 1 input paamete: SLLA is the souce link laye addess. 2. Context vaiable: LLA, counte; 3. Time: T I = 1s, T D = 5s, T P = 1s, T R [30, 45](s) 4. State: N NONE; I INCOMPLETE; S STALE; R REACHABLE; D DELAY; P PROBE. STATE_COVER Test NUD FSM: state cove 6 TRANS_COVER Test NUD FSM: tansition cove 36 NUD_Recv_NS Test NUD FSM: when eceiving an NS 8 Total Numbe 91