Testing Industrial VoIP Implementations: Registration for SIP Phones

Transcription

1 Testing Industrial VoIP Implementations: Registration for SIP Phones James J. Allen, Jr. William H. Chriss, Eva B. Phillips, Diane Somers Shahank Sarwate Avaya Inc., Lincroft, New Jersey M. Ümit Uyar, Samrat Batth City College and Graduate Center of the City University of New York 1

2 Modeling SIP Registration for Automated Test Sequence Generation 2

3 Framework for Test Generation and Test Execution Specification FSM Model of the Spec Minimum Cost Test Sequence RFC 3261 RFC 3665 RFC 3666 Etc. Manual Process POSTMAN Implementation Under Test (IUT) Executable Test Scripts Manual Process ATEE ATEE 3

4 SIP Registration Example from Understanding SIP by A.B. Johnston Figure 2.3 SIP registration example 4

5 Registration Model for SIP using UDP Transport Registration Model For UDP Transport 5

6 Postman + ATEE Environment for Automated SIP Conformance Testing 3-way Conference Testing Setup Button Depress Unit Button Depress Unit Button Depress Unit Alice s SIP Phone (under test) Bob s SIP Phone (under test) Carol s SIP Phone (under test) Ethernet RS232 Ethernet Hub Alice is the initiator of 3-way Conference calls Bob and Carol are the 2 nd and 3 rd legs of Conference Every message from/to 3 SIP phones are monitored No Proxy nor a Registrar (TOTAL CONTROL and OBSERVATION by TEST SCRIPTS) Automated Test Execution Engine (using test scripts based on Postman output) 6

7 SIP Conference FSM Model : Edges covered by the example scenario : 6 out of 400 steps, or 40 out of 5,000+ transactions from Postman 7

8 Guidelines for Test Engineers and Conclusions 8

9 SIP Registration Example from Understanding SIP by A.B. Johnston Figure 2.3 SIP registration example : Edges covered by registration example : 4 out of 400 transactions from Postman 9

10 Complexity of SIP Testing: It is a lot harder than it may seem Situations presented in RFCs cover only a few percent of Postman model coverage Test sequences generated by Postman: Registration: 500+ transactions for UDP Registration: 400+ transactions for TCP Conference: 5,000+ transactions Hold: 4,000+ transactions 10

11 Conformance Testing Value Market size opportunity depends on successful interoperability with the many proxy vendors. Completeness of conformance testing can lead to lower maintenance costs. Standards completeness is not essential, but adds significantly to complexity. Completeness in testing is not intuitive. Tools are essential to test case creation. 11

12 Test Coverage of SIP Testing: Complete control and observation by SIP test system Using an off-the-shelf Proxy for testing SIP phones is insufficient: cannot generate timeouts, inopportune and invalid inputs to test the SIP phone such cases will happen in the field and SIP phone s response will be untested A test system with complete control and observation over SIP phone is required: must generate any Proxy behavior as test scripts dictate 12

13 ETSI SIP Test Suite ETSI SIP Test Suite covers: Registration: 27 test cases Registrar: 27 test cases Call control (valid, timer, invalid): 172 test cases Proxy (valid, timer, invalid): 208 test cases Querying and messaging (proxy and end point): 173 test cases 13

14 ETSI SIP Test Suite Avaya plans to pursue the tests generated for SIP as complementary tests to the existing ETSI test suite. Typically every ETSI test case is analogous to an edge in the FSM models of Avaya. FSM models are much more comprehensive due to advanced automation capabilities of Avaya 14

15 About the Speaker James J. Allen, Jr. The leader of a group with 30 members, specializing testing and verification of Avaya telephones. Industrial experience: Bell Labs: Distinguished Member of Technical Staff, Supervisor Supervised testing groups for ISDN telephones, digital telephones, and IP Telephones. Holds two U.S. patents in telephone design M.S. in Engineering Science, Purdue Univ., Indiana. 15

16 Disclaimers The FSM models, portions of test sequences, implementation information, and all other testing related material presented in this tutorial do not reflect those of the AVAYA products. 16