Milan Zoric, ETSI Sebastian Müller, FSCOM Scott Moseley, Farbum Scotus 1st TTCN-3 User Conference, Sophia Antipolis, May 2004

Transcription

1 Testing Wireless over Wire with TTCN-3 Milan Zoric, ETSI Sebastian Müller, FSCOM Scott Moseley, Farbum Scotus 1st TTCN-3 User Conference, Sophia Antipolis, May

2 Wireless over wire SUT Tester Prototype UT Virtual tester 2

3 Testing over the network ATS Test System TE + SUT IP Network 3

4 ETSI Has a Long History of Producing Test Specifications Conformance test specifications since early 90's GSM, DECT, VB5, HiperLAN2, VoIP, ISDN, INAP, TETRA, IP Cablecom, 3GPP Moved from strict regulatory testing, with expensive and cumbersome 3rd party testing regimes to lean test suites focusing on conformance for interoperability that are "extremely good value for money" Strong demands for technologies that have earlier used other approaches (IP testing) Now also producing interoperability test specifications SIP <-> H.323 interworking ETSI has unique resources related to testing TC MTS, The PTCC, The Plugtests TM Service 4

5 First challenge: start with multiple unstable components Product (SUT) Test system ETS Lower layers including radio Base Standard ATS (TTCN) 5

6 More challenges The radio based test tool May not be available at all Commercial reasons May become available late Commercial and technical reasons May be excessively expensive Real costs combined with limited number of potential vendors Test suite validation the process of assuring that the ATS not only passes through the compiler but also executes successfully More expensive done in the context of radio based test tool development Possibly available later than desired for in-house prototype debugging for providing feedback and making corrections to a stadard Before testing against conformance test tools or other implementations, each company individually spends a lot on in-house testing/debugging schemes 6

7 The goals Make test suite validation faster and cheaper Enable sharing of efforts (costs) in protocol conformance testing and company in-house testing/debugging Make product testing/certification available to all technologies with alternative solutions Cheaper and faster development of radio based protocol test tools, or Combination of some level of protocol conformance testing and interop events 7

8 The implementation goals Definition of a test tool implementation framework Abstract Fits with ISO 9646 methodology and TTCN-3 language Generic Reusable for different protocols Reusable for different transport mechanisms, including full radio based transport Tool independent Using standardized interfaces Extensible For modifications of the test suite and the implementation First virtual tester implementation Modular Easy to understand Demonstrate feasibility of the approach Maintainable 8

9 Protocol Layer Tester (PLT) ASPs Protocol Layer Tester PUT ASP(PUT PDUs) 9

10 PLT and PUT Components Tester Platform PUT Platform Test Execution Controller PUT Execution Controller Test Suite Codec Protocol Under Test (PUT) Codec Test Runtime Adapters PUT Adapter Transporter 10

11 Transport mechanism PLT Platform Test Runtime Adapters API PUT Platform PUT Adapters Interface Wire Interface TE Transport Module PUT Transport Module 11

12 Generic wire based test tool PC Test Execution Tool ATS Message Snapshot Wire connection Wire Transport module Generic approach to message snapshot content Generic definition of the Application Programming Interface Wire API 12

13 Specific wire based test tool PC TTCN-3 Test Execution Environment ATS HA Datagram UDP/IP on wire HA Adaptation Layer HA API HA specific message snapshot content HA specific Application Programming Interface 13

14 Generic API specification <<interface>> org::etsi::ttcn:udp::datagramsocketapi close(): void connection(in remote: SocketAddress): boolean disconnect(): void isconnected(): boolean receive(in packet: DatagramPacket, in timeout: int): boolean send(in packet: DatagramPacket): boolean <<interface>> org::etsi::ttcn:udp::datagrampacket getdata(): byte[] getlength(): int getoffset(): int getsocketadress(): SocketAddress setdata(in buf: byte[], in offset: int, in length: int): void setdata(in buf: byte[]): void setlength(in length: int): void setsocketaddress(in address: SocketAddress): void <<interface>> org::etsi::ttcn:udp::socketaddress gethostname() getport() setbzhostname() setport() 14

15 Specific API part <<interface>> org::etsi::ttcn::udp::datagrampacket <<interface>> org::etsi::ttcn::ha::hadatagrampacket setframecounter(in int value):void getframecounter():int setcontrolzonevalid(in boolean value):void getcontrolzonevalid():boolean setsidvalid(in boolean value):void getsidvalid():boolean... setbody(in byte[] value):void getbody():byte[] 15

16 Testing one protocol entity HA PUT : AP UDP/IP on wire PLT : AT 16

17 Testing multiple protocol entities HA PUT : AP1 UDP/IP on wire PLT : AT HA PUT : AP2 17

18 First use of the test system prototype Revealed Errors in prototype implementation Ambiguity in specification Defects in Abstract Test Specification The above are assumptions to be clarified and corrected between standard writers, implementers and test specification developers 18

19 TTCN-3 Execution Environment Test System User TCI TM: Management CH: Component Handling control behaviour componentste types / data ports timer CD: Codec TRI SA: System Adaptor PA: Platform Adaptor SUT: System Under Test 19

20 The Test Implementation Framework Implementation of TTCN-3 send() TE calls trisend() in SA TE Sending of encoded PDU TRI SA Building of API Message Depends on the type of PDU Contains various information in the header UDP HDR PDU MAC ID Long vs. Short Channel PDU Up-/Downlink SUT Sending to System Under Test 20

21 Construction of Datagrams UDP HDR PDU 21

22 Using the radio link Implementation of TTCN-3 send() TE calls trisend() in SA Sending of encoded PDU Using the lower layer SAP Introduces framing Power level, frequency etc. Sends it out using radio Sending to System Under Test Test Device TE SA PDU... SUT 22

23 Test validation against the SDL model block type RRC_AP_UDP_BT UDP_1:UDP_PT api_in udp2 api_out UDP_sock.con, UDP_resolv.ind, UDP_close.ind, UDP_bindok, UDP_bindfail, pong_udp_receive UDP_sock.req, UDP_resolv.req, UDP_bind, UDPsignalRoute 1(2) pong_udp_prep, pingasn_udp_send UDPgate UDP_sock.con, UDP_resolv.ind, UDP_close.ind, UDP_bindok, UDP_bindfail, UDP_sock.req, UDP_resolv.req, UDP_bind, pong_udp_prep, pingasn_udp_send API API_DL API_UL API pong_udp_receive api_dl api_ul rrcdl API_handler: API_handlerType Bcast rrcul (RRC_DL) RRC_DL RlcGeneralBroadcastInformation BcastRoute RRC_UL (RRC_UL) RRC_RLC_out B RRC_RLC_in RRC_AP: RRC_AP_PT 23

24 Results Test system prototype Using current version of the test suite Using only standardized interfaces Using the generic test implementation framework Easy adaptation to test devices, using other lower layers Implementation will be made available to the indust Abstract Test Suite Validated Executable Test Suite running in the Test System over UDP/IP Test Management for execution Documentation 24

25 New challenges Timing Requirements on timing of all testing related activities may be different and rather challenging There may be implications on how test specifications are written Implementation issues that affect prototype testing/debugging Software/hardware division in relation to API positioning 25

26 Acknowledgements EP BRAN (Broadband Radio Access Networks) for their support EC for funding part of the work Member companies that provided protocol stack executables to be tested Testing_tech for providing the TTCN-2/TTCN-3 converter, TTCN-3 compiler and run time environment Kaiserslautern University for providing apigen software and extensive support Prof. Dr. Reinhard Gotzhein, Marco Brandt Theofanis Vassiliou-Gioles for his contribution to this work Protocol test specifications developed by Gérard Daugan, Scott Moseley, Jean Claude Wattelet 26